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(1)  Biological  data  from  cat,  monkey,  and  man,  using  neurophysiological  and 
psychophysical  methods,  provide  much  evidence  for  the  existence  of  mechanisms 
tuned  to  different  regions  of  spatial  frequency  and  orientation.  In  general,  the 
spatial  frequency  bandwidth  is  one  to  two  octaves  and  the  orientation  bandwidth 
is  +15  degrees.  The  overall  filtering  characteristics  of  the  visual  system 

is  the  envelope  of  the  activity  of  those  mechanisms  and  can  be  determined  by  con¬ 
trast  sensitivity  and  contrast  matching  functions. 

(2)  Two-dimensional  spatial  filters  were  made  based  upon  the  general  and 
specific  filtering  properties  of  the  overall  (MTF)  and  individual  mechanisms 
(channels).  These  spatial  filters  were  used  to  filter  a  wide  variety  of  objects. 

(3)  The  perceptual  problems  of  the  classification  of  simple  and  complex 
objects.  Gestalt  principles,  visual  textures,  visual  illusions,  and  multi¬ 
stability,  with  few  exceptions,  yielded  gracefully  to  the  concept  of  spatial 
filtering.  Parsimony  reigned  as  explanations  for  these  seemingly  disparate 
visual  phenomena  were  given  in  terms  of  the  known  two-dimensional  filtering 
properties  of  the  visual  system. 

(4)  A  detailed  study  using  generalized  and  biologically  derived  band-pass 
spatial  filters  further  demonstrated  the  power  of  a  general  filter  concept  for 
visual  perception.  The  full  spectrum  of  perceptual  information  about  a 
complex  object  from  basic  form  to  the  finest  detail  was  available  with  only 
seven  two-octave-wide  band-pass  filters  (channels).  These  results  led  to  the 
concept  of  a  hierarchy  of  filtered  images. 

(5)  In  general,  the  MTF  filter  provided  the  details  of  objects  that 
corresponded  to  what  is  seen.  One-to  two-octave  band-pass  filters  appeared  to 
provide  the  generalized  information  that  is  perceived  about  objects.  The 
specific  bandwidth  and  shape  of  the  spatial  filters  were  not  important  for  the 
class  of  objects  and  perceptual  problems  discussed. 

(6)  Combining  the  filtering  results  of  the  minimum  information  required  for 
the  identification  of  objects  with  data  from  an  experiment  that  determined  the 
minimum  amount  of  contrast  required  for  the  identification  of  Snellen  letters 
demonstrated  the  predictive  power  of  the.  filter  concept.  The  Snellen  acuity  of 
17  of  22  eyes  of  amblyopes  and  patients  suffering  from  multiple  sclerosis  could 
be  predicted  within  one  Snellen  line.  The  increase  of  spatial  sensitivity  over 
a  narrow  range  of  spatial  frequency  of  one  amblyope  resulted  in  a  concomitant 
increase  in  Snellen  letter  acuity.  Especially  interesting  was  that  the  increased 
Snellen  acuity  could  be  explained  from  the  minimum  number  of  spatial  frequencies 
required  for  letter  identification.  These  results  appear  to  provide  the  first 
direct  evidence  for  the  existence  of  narrow-band  mechanisms  in  the  visual  system 
which  are  used  in  the  identification  of  complex  objects.  Thus,  the  ability  to 
establish  a  relationship  between  Snellen  letter  acuity  and  contrast  sensitivity 
suggests  that  the  filter  concept  is  neither  blind  nor  empty. 

In  general,  these  results  support  the  view  that  the  MTF  is  the  window  which 
reduces  the  range  of  size  of  objects  that  can  be  seen.  The  much  narrower  and 
orientation-selective  filters  that  make  up  the  MTF  appear  to  filter  relevant 
form  information.  The  outputs  of  all  the  individual  filters  appear  to  be 
presented  simultaneously  to  create  the  global  picture  that  is  seen. 

The  picture  that  emerges  from  these  studies  is  that  visual  perception  may  be 
structured  from  a  hierarchy  of  filtered  images,  each  image  representing  one  level 
of  visual  information.  This  hierarchy  of  filtered  images  may  be  the  keyboard 
from  which  the  harmony  of  visual  perception  is  obtained. 
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PREFACE 


This  report  is  a  dissertation  submitted  to  the  University  of  Cambridge  for  a  Ph.D.  in  Biophysics  which  is  being  made 
available  to  the  larger  research  community.  Alluding  to  early  advances  in  Greek  arts  and  sciences,  John  Platt 
categorized  the  two  general  classes  of  scientific  minds  as  Dionysian  and  Apollonian.  Although  dissertations  can  be 
typically  categorized  as  Apollonian  (since  novel  major  thrusts  in  science  can  be  risky  and  quite  unsettling  to  the 
scientific  status  quo),  this  dissertation  is  primarily  Dionysian.  It  offers  a  new  synthesis  of  visual  perception  within  a 
single  conceptual  framework — filtering — to  explain  how  we  see  objects.  It  iB  a  different  way  of  thinking  and  quantifying 
perception  not  found  in  contemporary  texts.  It  attempts  to  provide  a  logical  bridge  between  current  knowledge  about 
visual  science  and  perception  using  engineering  systems  analysis. 

General  theories  about  perception  do  not  usually  consider  the  physics  of  the  visual  system — how  the  great  masses  of 
spatial  information  are  physically  transmitted  up  the  visual  pathway.  Since  the  visual  system  is  a  physical  processing 
system,  it  is  reasonable  to  suggest  that  the  physical  filtering  properties  of  the  mechanisms  that  encode  and  transmit 
spatial  information  determine  to  a  large  extent  the  kind  of  processing  that  can  occur.  It  seems,  therefore,  crucially 
important  to  understand  the  filtering  characteristics  of  those  mechanisms.  This  means  that  the  bandwidth,  center 
frequencies,  and  weighting  functions  of  the  filters,  their  degree  of  linearity,  and  how  they  interact  must  be  determined 
to  fully  understand  how  they  create  the  global  picture  that  we  see.  Thus  the  general  approach  here  is  to  use  such  data 
to  filter  objects  to  determine  what  kinds  of  information  can  be  provided  by  such  a  system  of  filters.  Our  ordinary 
perception  of  objects  becomes  the  boundary  conditions  to  which  the  filtered  images  are  compared.  In  this  way.  perhaps 
a  physical  foundation  for  perception  can  be  created. 

It  became  clear  at  the  beginning  of  this  research  that  one  could  easily  devote  full  time  to  one  small  aspect  of 
perception — illusions,  for  example.  However,  the  successful  modelling  of  one  small  aspect  of  perception  does  not 
guarantee  that  the  insights  and  results  of  the  model  can  collapse  across  all  other  perceptual  problems  within  that  single 
framework.  Since  there  exists  no  general  quantitative  theory  of  perception,  it  seemed  crucial  in  the  early  stages  to 
extend  the  concept  of  filtering  to  many  perceptual  areas  as  quickly  as  possible  to  test  the  parsimony  of  the  notion.  The 
first-order  approximation  of  results  was  sought— 90%  of  the  solution.  Thus  the  major  criteria  set  for  the  results  of  each 
stage  of  this  work  were:  are  these  results  internally  and  externally  consistent  with  one  another  and  with  current 
knowledge  in  visual  science,  and  are  they  testable?  Once  those  criteria  were  achieved  for  each  area  of  perception, 
parsimony  could  be  tested  rapidly  while  simultaneously  maintaining  scientific  credibility.  Relating  the  ability  of  the 
filtering  characteristics  of  the  visual  system  to  predict  the  detection  and  identification  of  complex  objects  of  persons 
having  normal  and  abnormal  vision  represents  an  attempt  to  tie  earlier  filtering  results  into  direct  measurements  of  the 
filtering  properties  of  the  mechanisms  of  the  visual  system.  Time  will  tell  whether  this  approach  continues  to  prove 
helpful. 

There  is  no  attempt  in  this  dissertation  to  argue  every  point  and  offer  alternatives  to  each  possible  explanation  for  each 
demonstration.  There  are  many  questions  that  still  remain  unanswered.  The  reason  for  this  approach  is  simple.  It  is 
almost  impossible  (and  not  desirable  at  our  state  of  ignorance)  to  travel  down  each  stream  while  trying  to  reach  a  source 
of  a  river.  Only  intuition  can  be  used  to  guide  such  travel.  Thus  the  intuition  derived  from  the  preceding  research 
directed  the  next  step.  Since  one  person's  path  is  another's  maze,  this  approach  will  not  please  all  readers.  Indeed,  it 
would  be  disappointing  if  it  did,  for  it  would  represent  agreement  in  areas  full  of  differences,  hardly  representative  of  a 
fresh  approach  to  long-standing  problems. 

This  strategy  means  that  the  explanation  for  certain  phenomena  written  here  is  not  suggested  to  be  the  only  possible 
explanation.  However,  the  explanations  are  internally  consistent.  Since  there  was  no  other  general  model  to  test  these 
ideas  against,  no  other  approach  seemed  tenable  at  that  time.  Science  is  the  best  story  that  is  constructed  from  all  the 
currently  known  facts.  It  will  be  quite  surprising  if  most  interpretations  here  stand  for  long  periods  of  time  in  terms  of 
completely  explaining  how  each  perceptual  phenomenon  occurs.  This  research  represents  the  beginning  of  a 
quantitative  approach  to  understanding  the  important  spatial  information  in  objects  for  various  aspects  of  perception. 
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Since  this  work  attempts  to  replace  words  by  numbers,  it  offers  an  approach  to  many  applied  problems.  The  transfer  of 
this  technology  from  basic  to  applied  research  areas,  applicable  to  certain  military  and  civilian  problems  in  vision,  is 
currently  under  way.  It  is  hoped  that  this  work  will  help  to  create  a  firm  foundation  for  such  applied  areas  as  display 
design  and  object  detection  and  identification  by  man  and  machine.  Since  this  research  uses  the  same  mathematical 
language  to  describe  the  filtering  properties  of  the  observer  and  the  spatial  information  of  objects  that  are  being  used  to 
describe  certain  sensor  and  display  parameters,  it  may  provide  the  basis  for  unified  metrics  in  those  areas.  The  results 
of  the  next  series  of  experiments  attempting  to  transfer  this  technology  will  determine  the  effectiveness  of  this  approach. 

No  research  can  be  done  effectively  in  a  complete  vacuum.  It  wasn't  until  I  received  feedback  about  my  early  work  on 
filtering  and  visual  perception  in  1471  and  1972,  after  initial  lectures  at  the  Neurosciences  Research  Program  in  Boston 
and  The  Physiological  Laboratory  in  Cambridge,  England,  and  other  scientific  institutions  that  I  became  aware  of  the 
work  done  by  others  in  psychophysics  and  neurophysiology  that  supported  some  of  the  original  filtering  results.  This 
dissertation  represents  the  major  integration  and  extention  of  those  early  filtering  results  into  contemporary  visual 
science. 

Thus,  there  are  many  individuals  to  whom  1  would  like  to  express  my  gratitude:  Dr.  D.  Pollen,  who  first  took 
an  interest  in  my  work,  and  Dr.  R.  Held,  whose  comment  on  the  filtering  after  my  first  lecture:  "It  is  an  idea  whose 
time  has  come, "  helped  spur  me  on  to  the  next  stages.  Similarly,  Dr.  F.  Campbell,  also  provided  early 
encouragement  and  acted  as  my  research  supervisor  at  Cambridge.  I  am  also  grateful  to  Dr.  J.  Robson,  Dr.  D. 
Tolhurst,  I)r.  C.  Furchner,  Dr.  R.  Sekuler.  Dr.  J.  Mollon,  Dr.  J.  Foley,  and  Dr.  H.  Howland  for  their  interesting 
discussions  at  various  stages  of  this  work.  For  the  technical  assistance  par  excellence  of  Clive  Hood  at  Cambridge  goes  a 
special  thank  you.  I  have  been  fortunate  enough  to  have  been  invited  to  lecture  on  the  early  results  of  this  work  in  the 
United  States.  Great  Britain.  Europe,  and  the  L.S.S.R.  I  have  benefited  greatly  from  that  exposure  and  thank  the 
numerous  individuals  for  their  interest  and  criticism. 

The  assistance  provided  by  Dr.  R.  Zimmern.  Ms.  R.  Banks,  and  Ms.  J.  Lowry  of  Addenbrooks  Hospital.  Cambridge, 
for  securing  and  examining  patients  used  in  the  clinical  studies  reported  in  this  dissertation  is  gratefully  acknowledged. 

I  am  also  grateful  to  Dr.  <).  Saxton.  Dr.  D.  M.  Horner,  and  Mr.  T.  Pitt  of  the  Electron  Microscopy  Branch  of  Applied 
Physics  at  the  University  of  Cambridge  for  sharing  their  SEMPER  system  with  me. 

Dr.  M.  Cannon,  Dr.  M.  Nelson.  Mrs.  A.  Elshoff,  and  Mrs.  M.  Whitaker  are  thanked  for  their  help  with  the  final 
editing  and  proofing.  I  would  also  like  to  thank  Ms.  F.  Hake  for  the  illustrations  and  Mr.  P.  Starling  and  Mr.  R. 

Brooks  for  photographing  the  figures.  I  am  also  grateful  to  Ms.  J.  Jackson  who  typed  the  final  copy  of  this  dissertation. 
Thanks  also  to  Mr.  S.  Fullenkamp  and  Lt.  D.  Evans  for  help  with  the  logistics  of  putting  this  report  physically 
together. 
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INTRODUCTION 


1 

The  everyday  objects  that  we  see  contain  a  tremendous  amount  of  information.  Our  remarkably  sensitive  eye  uses 
more  than  1 10,000,000  rods  and  about  6,000,000  cones  to  sample  objects  into  at  least  100  different  levels  of  brightness 
and  about  50  distinguishable  colours.  It  is  highly  unlikely  that  the  visual  system  can  process  all  that  information. 

Indeed,  any  real  system  must  have  limits  as  to  the  amount  of  information  it  can  process.  As  will  be  shown,  it  is  the 
nature  of  the  visual  system  to  throw  away  much  of  the  detailed  information  about  objects.  One  way  to  describe  the  data 
reduction  is  in  terms  of  filtering,  that  is,  the  selection  of  a  certain  range  of  a  particular  physical  property. 

Kenneth  Craik  1 1%6)  suggested  as  a  general  principle  that  a  filter  is  needed  to  separate  one  object  from  another: 

"The  essential  part  of  physical  ‘recognizing’  instruments  is  usually  a  filter  —  whether  it  be  mechanical  sieve,  an  optical 
filter,  or  a  tuned  electrical  circuit  —  which  passes  any  quantities  it  is  required  to  identify  and  rejects  all  others. " 

One  filter  alone  cannot  provide  the  information  to  perform  all  the  tasks  required  for  perception.  Several  filters  are 
required  and  they  cannot  be  too  broad  or  too  narrow.  As  Craik  noted:  “Our  ordinary  perception  hovers  uncertainly 
between  overgeneralization,  which  would  lead  us  to  ignore  initial  differences  between  objects,  and  overspecificity, 
which  would  lead  us  never  to  recognize  the  identity  of  objects  unless  they  were  in  every  detail  indistinguishable;  the 
first  would  end  us  in  confusion,  the  second  in  inability  to  draw  general  conclusions  about  the  world  we  see."  Applying 
these  concepts  to  object  recognition  implies  at  least  one  filter  to  capture  the  general  form  of  an  object  and  another 
filter  to  capture  the  details. 

The  application  of  filtering  to  perception  suggests  that  certain  information  about  objects  must  be  redundant  for  certain 
tasks.  Here,  redundant  means  information  that  is  extra,  not  needed  for  the  task.  For  example,  serifs  found  on  letters 
are  redundant  for  the  classification  of  the  letter.  However,  information  about  the  serif  is  required  if  the  type  of  font  of 
the  letter  is  to  be  determined.  The  importance  of  redundancy  in  visual  processing  has  been  stressed  by  Attneave  1 16541 
and  Barlow  1 1%1 1. 

The  first  parts  of  this  dissertation  deal  with  certain  fundamental  questions:  Can  the  concept  of  filtering  unify  the 
understanding  of  visual  information  processing?  How  much  of  visual  perception  can  be  accounted  for  by  the 
information  contained  in  mechanisms  that  inevitably  lose  information  in  the  filtering  process?  To  answer  these 
questions,  many  of  the  important  general  phenomena  of  perception  must  be  considered:  object  recognition,  illusions, 
textures,  and  niultistable  objects.  A  general  theory  of  visual  perception  should  encompass  all  of  these 
phenomena  within  a  unified,  quantitative  structure.  Such  a  theory  will  be  attempted  in  this  dissertation.  However, 
in  order  to  provide  realistic  interpretations  of  how  the  visual  system  could  be  processing  spatial  information,  certain 
constraints  must  be  imposed. 

The  main  constraint  on  the  filtering  presented  here  is  that  it  be  based  upon  the  filtering  properties  of  known 
mechanisms  in  the  visual  system.  This  constraint,  however,  begs  the  question:  What  is  known  about  the  filtering 
properties  of  the  visual  system?  A  review  of  the  neurophysiological  and  psychophysical  literature  reveals  much 
information  about  the  spatial  filtering  characteristics  of  mechanisms  in  the  visual  system.  Although  a  wealth  of 
data  is  available,  it  has  not  been  integrated  into  a  general  understanding  of  visual  information  processing.  Thus,  the 
first  parts  of  the  dissertation  will  attempt  to  answer  the  question:  What  could  the  large  amounts  of  data  collected  over 
the  last  fifteen  years  about  the  spatial  processing  of  the  visual  system  explain  about  the  perception  of  everdav  objects? 

As  the  filtering  properties  of  the  visual  system  are  examined,  it  will  be  shown  that  the  spatial  information  that  is 
selectively  lost  by  certain  mechanisms  determines  the  ap|*eurance  of  objects  under  certain  viewing  conditions.  The  use 
of  filters  based  on  biological  data  will  make  it  possible  to  see  objects  as  the  various  stages  of  visual  processing  would 
"see"  them.  The  information  that  remains  after  certain  stages  of  data  reduction  can  be  shown  to  account  for  a  variety  of 
phenomena  including  the  classification  of  patterns  that  range  from  letters  to  faces.  Gestalt  law«  of  perceptual 
organization,  many  geometric  and  contrast  illusions,  the  isolation  of  form  from  texture,  and  niultistable  objects. 

Furthermore,  a  limited  range  of  information  will  be  shown  to  be  sufficient  for  form  generalization.  Indeed,  it  will  be 
show  n  that  only  a  few  ripples  from  the  tidal  wave  of  information  that  washes  over  the  retina  are  needed  to  account 
for  much  of  how  we  see  objects. 


10 


*  V.i  rh 


The  later  chapters  in  this  dissertation  relate  a  conventional  measure  of  visual  acuity  to  a  psychophysical  measure  that 
provides  more  detailed  and  quantitative  information  about  the  filtering  characteristics  of  the  visual  system. 
Experimental  data  are  used  to  offer  an  explanation  of  why  certain  filtering  characteristics  of  the  visual  system  will  not 
be  detected  by  conventional  measurements  of  visual  acuity.  Further,  these  data  provide  additional  quantitative  support 
for  the  notion  that  relatively  little  spatial  information  is  required  to  correctly  identify  complex  objects — a  major  finding 
of  the  earlier  chapters  of  this  dissertation. 

Appendix  A  contains  the  details  of  experiments  on  abnormal  vision  that  were  used  to  test  the  ability  of  certain  data  to 
predict  a  conventional  measure  of  visual  acuity.  These  data  were  collected  in  an  attempt  to  provide  more  insight  into 
visual  processing  by  investigating  the  filtering  characteristics  of  abnormal  vision.  In  particular,  studies  of  one  amblyope 
showed  highly  tuned  mechanisms  in  each  eye  that  have  not  been  revealed  before  and  that  appear  to  compensate  for 
the  different  filtering  characteristics  of  each  eye  under  certain  circumstances.  Further  studies  with  patients  suffering 
from  multiple  sclerosis  provided  filtering  characteristics  that  substantiate  complaints  of  poor  vision  which  appears 
normal  when  conventional  acuity  measurements  are  used. 

In  summary,  this  dissertation  presents  the  argument  that  the  concept  of  spatial  filtering  can  provide  a  quantitative 
and  parsimonious  means  of  explaining  many  phenomena  of  spatial  vision.  It  will  be  argued  that  it  is  impossible  for 
the  visual  system  to  process  all  the  information  available  about  the  objects  that  are  seen.  The  reduction  of  spatial 
information  that  occurs  at  various  stages  of  processing  in  the  visual  system  can  be  examined  by  filtering  objects  using 
spatial  filters  based  on  biological  data  about  the  shape  and  size  of  certain  visual  mechanisms.  The  consequences  of 
such  filtering  provide  images  that  relate  to  how  objects  are  seen  and  suggest  that  a  major  role  of  the  organization  of 
the  visual  system  is  to  extract  information  from  a  relatively  small  numbered  hierarchy  of  filtered  images. 

First,  however,  some  mathematical  theory  that  will  be  needed  to  understand  the  relationship  between  filtering  and 
visual  perception  will  be  presented. 


PART  I 


A  THEORETICAL  BACKGROUND  TO 
FOURIER  ANALYSIS  AND  SPATIAL  FILTERING 


“Man  tries  to  make  for  himself  in  the  fashion  that  suits  him  best  a  simplified  and  intelligible  picture  of  the  world 
The  supreme  task  .  is  to  arrive  at  those  universal  elementary  laws  from  which  the  cosmos  can  be  built  up  by  pure 
deduction.  There  is  no  logical  path  to  these  laws;  only  intuition,  resting  on  sympathetic  understanding  of  ex|>erienre. 
can  reach  them  .  .  .  " 


Albert  Einstein 
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CHAPTER  1 


LINEAR  SYSTEMS  ANALYSIS 


1.1  Introduction 

The  use  of  Fourier  analysis  by  visual  scientists  is  becoming  increasingly  widespread.  However,  it  is  important  to  point 
out  how  Fourier  analysis  fits  into  the  general  and  powerful  framework  of  linear  systems  analysis.  This  understanding  is 
necessary  in  order  to  appreciate  why  Fourier  analysis  is  unique  in  the  spatial  analysis  of  complex  objects.  A  brief 
discussion  of  systems  analysis  that  leads  to  the  Fourier  transform  and  spatial  filtering  may  aid  this  understanding. 

Some  aspects  of  this  discussion  are  amplified  in  Cooper  and  McGillem  1 1%7). 

The  Fourier  transform  will  play  three  roles:  first,  as  a  quantitative  description  of  the  spatial  information  of  objects: 
second,  as  a  means  to  filter  the  spatial  information  of  objects;  third,  as  a  general  model  of  how  the  visual  system 
ap|iears  to  be  processing  local  and  global  spatial  information.  It  is  extremely  important  for  the  reader  to  be  aware  of 
how  the  Fourier  transform  is  being  used  throughout  this  dissertation. 

A  system  consists  of  mechanisms  that  are  assembled  and  interact  with  one  another  to  perform  some  function.  For 
example,  the  eye-brain  system  is  composed  of  mechanisms  assembled  to  perform  the  function  of  detecting  and 
classifying  objects.  The  Usds  that  have  been  developed  to  analyze  systems  constitute  systems  analysis.  The  major 
problem  of  system  analysis  is  to  define  the  response  of  a  system  when  the  system  and  the  input  are  known.  The 
solutions  to  this  problem  provide  predictive  power  about  the  behaviour  of  the  system  under  certain  conditions.  One 
part  of  this  problem  is  the  determination  of  a  mathematical  description,  or  model,  for  the  system  which  allows  the 
predictions  to  be  quantitative. 

1 .2  Linearity.  Nonlinearity,  and  the  Power  of  Superposition 

Two  of  the  terms  used  to  classify  systems  are  linear  and  non-linear.  A  linear  system  obeys  the  law  of  superposition.  If 

R|lxl  —  system  response  to  stimulus  S  |  lx  I 
Rjlxl  —  system  response  to  stimulus  Sjlxl 

and 

AH  1 1 x I  +  BRjIxl  =  system  response  to  AS|(xl  +  BS2<xl 

for  all  A.  B.  S  |lx)  and  Sjlxl.  then  the  system  is  linear.  If  this  is  not  true,  then  the  system  is  nonlinear.  The  power  of  the 
law  of  superposition  comes  from  allowing  calculations  of  the  response  of  a  system  to  complex  stimuli  simply  by  taking 
the  sum  of  the  res|>onses  of  a  system  to  individual  simple  stimuli. 

All  real  systems  have  some  degree  of  nonlinearity.  However,  the  nonlinearities  can  be  ignored  if  they  are  small  in 
comparison  to  the  overall  operating  characteristics  of  the  system.  If  the  nonlinearity  cannot  be  ignored,  then  a 
piecewise  analy  sis  is  used  in  which  each  section  of  the  response  is  approximated  by  a  linear  function. 

In  probing  an  unknown  system,  linear  analysis  is  very  powerful.  The  deviation  from  linearity  can  be  used  to  determine 
the  amount  of  nonlinearity  in  the  sy  stem. 

1 .11  Periodic  and  Nonperiodic  Stimuli 

The  input  to  a  system  is  the  stimulus.  For  the  visual  system,  the  stimulus  is  a  pattern  of  light  intensity  that  varies  over 
space  (excluding  temporal  aspects  for  the  moment).  Pattern  and  object  will  be  used  somewhat  interchangeably  as  loose 
descriptions  of  the  stimuli  in  this  dissertation.  In  general,  pattern  will  refer  to  a  simple  stimulus,  whereas  object  will 
refer  to  a  more  complex  stimulus.  In  order  to  carry  out  a  systems  analysis,  a  mathematical  representation  of  the 
pattern  is  used. 

Patterns  may  be  classified  as  periodic  or  nonperiodic.  A  periodic  pattern  is  one  that  repeats  the  sequence  of  values 
over  a  fixed  distance,  called  the  period.  More  precisely,  a  pattern,  plxl.  is  periodic  if  there  is  a  number  X  such  that 


plx)  sb  plx  +  X)  for  all  x;  all  other  patterns  are  nonperiodic.  The  smallest  positive  number  that  satisfies  the  previous 
relationship  is  the  period,  X,  which  also  defines  the  duration  of  one  complete  cycle  of  the  pattern.  Integral  submultiples 
of  X  are  called  harmonics.  The  fundamental  frequency,  f  j ,  of  a  periodic  pattern  is  inversely  related  to  the  distance  X 
where  f  |  =  1/X. 

Sinusoids  are  an  example  of  periodic  patterns.  As  shall  be  shown,  however,  even  nonperiodic  patterns  can  be 
represented  in  terms  of  periodic  functions.  From  the  standpoint  of  the  mathematical  representation  of  patterns,  the 
(teriodic  class  is  of  considerable  theoretical  importance. 

1.4  Basis  Functions 


Most  patterns  observed  are  not  periodic.  Therefore,  a  set  of  periodic  functions  is  needed  with  which  to  represent  both 
periodic  and  nonperiodic  patterns.  To  create  a  quantitative  description  of  a  pattern.it  is  necessary  to  represent  it  in 
terms  of  explicit  space  functions  whose  numerical  values  are  exactly  defined.  These  considerations  raise  an  important 
question:  What  criteria  should  govern  the  selection  of  the  space  function?  There  are  many  different  ways  in  which  this 
representation  might  be  realized  with  the  same  objective.  The  choice  of  any  particular  method  is  usually  dictated  by 
mathematical  convenience. 


Mathematical  convenience  usually  suggests  that  a  pattern  plx  I  be  represented  as  a  linear  combination  of  a  set  of 
elementary  space  functions  X  n(xl.  These  elementary  functions,  called  basis  functions,  are  selected  to  have  certain 
convenient  properties. 

A  set  of  basis  functions  is  given  by  X()|xl,  Xjlxl,  AMxl,  ..  .  ,  Xn(xl.  where  n  can  range  to  infinity  I »  I.  A  linear 
combination  of  basis  functions  can  be  written  as 
N 

Blxl=  £  anXn<x» 

n  =  0  111 

where  an  is  the  corresponding  coefficient  of  the  basis  function  X  n(x  I.  There  is  no  general  solution  to  the  selection  of  the 
best  set  of  basis  functions  X  nlxl  for  any  given  application.  However,  there  are  many  known  results  available  for 
functions  that  have  certain  properties  which  can  guide  the  particular  selection  in  any  given  application. 

1.4.1  Orthogonality 


( )ne  inqsirtant  property  for  a  set  of  basis  functions  is  called  finality  of  coefficients.  This  property  allows  the 
determination  of  any  given  coefficient  without  the  need  to  know  any  other  coefficient;  that  is.  the  value  of  each 
twfficient  an  is  independent  of  the  value  of  any  other  coefficient.  For  example,  more  terms  can  be  added  to  the 
representation  to  allow  greater  accuracy  without  making  any  changes  in  the  earlier  coefficients.  Finality  of  coefficients 
is  achieved  when  the  basis  functions  are  said  to  be  orthogonal  over  the  space  for  which  the  pattern  is  defined. 

The  condition  of  orthogonality  for  real  basis  functions  requires  that 

fx2 

I  'Xnlxl  Xk  dx  =  0  for  k  ^  n 

J  x, 

=  n  k  for  k  =  n  (21 

for  all  k  and  n  over  the  space  x  |  to  x*.  If  qk  =  I  for  *11  it.  then  the  basis  functions  are  orthogonal.  In  words.  Kquation 
2  simply  requires  that  the  basis  functions  have  no  components  in  common. 


When  the  basis  functions  are  orthogonal  and  real,  then  the  coefficient  an  can  be  found  by 


an 


X  „lxl  Bl x I  dx 


ttl 
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1 .4.2  Integral  Squared  Error 


There  is  another  property  of  orthogonal  functions  which  is  quite  important.  In  many  situations,  the  accuracy  of  the 
representation  of  a  pattern  when  only  some  and  not  all  the  terms  are  used  must  be  determined.  Any  real  pattern  stored, 
for  example,  in  a  digital  computer  lor  the  visual  system),  can  be  represented  only  by  a  finite  number  of  terms.  A 
practical  problem  which  arises  is  how  to  select  a  set  of  basis  functions  that  minimizes  the  error  of  representation  of  the 
pattern  for  a  limited  number  of  terms.  One  measure  used  is  called  the  integral  squared  error  (I  I,  the  integral  of  the 
square  of  the  differences  between  the  pattern,  p(xl,  and  its  approximation,  plxl,  given  by 

1  =  r  -  [plxl- p(x)]-dx  (41 

J  x, 

The  integral  squared  error  is  zero  only  when  plxl  =  plxl.  If  not,  then  I  is  always  positive,  and  the  smaller  it  is,the  better 
the  approximation.  It  can  be  shown  that  orthogonal  basis  functions  not  only  have  the  property  of  finality  of 
coefficients  but  also  minimize  the  integral  squared  error  of  the  representation,  e.g..  Cooper  and  McGillem  1 1%7). 

Other  criteria  could  be  used  to  select  basis  functions;  however,  they  take  us  beyond  the  scope  of  this  discussion. 

So  far,  only  a  general  discussion  of  basis  functions  has  been  provided  and  no  suggestions  about  the  most  desirable 
types  of  functions  to  use  have  been  offered.  The  requirement  of  orthogonality  does  little  to  restrict  this  selection 
since  many  different  functions  can  be  made  orthogonal  by  suitable  definitions.  Therefore,  the  selection  of  basis 
functions  depends  primarily  upon  how  the  functional  representation  is  to  be  used.  In  systems  analysis,  sinusoids  are 
very  useful  functions  because  they  remain  sinsuoids  after  the  performance  of  various  mathematical  operations  needed 
in  such  an  analysis.  Addition  and  subtraction  of  two  sinusoids  of  the  same  frequency  results  in  a  sinusoid.  The 
derivative  and  the  integral  of  sinusoids  are  sinusoids.  These  properties,  combined  with  superposition,  suggest  that 
representing  a  pattern  as  a  sum  of  sinusoids  can  be  quite  convenient  as  well  as  powerful. 


CHAPTER  2 


FOURIER  ANALYSIS 


2. 1  The  Trigonometric  Fourier  Serif* 

The  power  of  using  sinusoids  to  represent  complex  functions  was  first  shown  by  the  French  physicist  Fourier 
1 1768- 18301  who  used  them  to  solve  problems  in  heat  transmission.  Out  of  his  work  has  come  Fourier's  theorem: 


( 2TT  \  (2  71  \ 

flxl  =  Co  +  C  |  cos  I  x  +(J>  l  I  +  Cj  cos  I  I 


+ 


(2  71 

x£s+* 


n) 
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where  Cp,  ,  Cn.  are  constants.  Fourier's  theorem  states  that  a  periodic  function  fix  I,  with  a  spatial  period  X,  can 
be  synthesized  by  a  sum  of  harmonic  functions  that  are  integral  submultiples  of  X  and  have  a  phase  angle  given  by  (b- 
The  accuracy  of  this  representation  of  a  function  is  increased  by  increasing  the  number  of  terms.  In  order  to  maintain 
the  proper  phase  relationship  between  the  harmonics  and  yet  remove  the  cumbersome  phase  term  in  each  harmonic,  a 
trigonometric  identity  is  used: 


Cn  cos  Inkx  +<J>nl  =  An  cosnkx  +  B„  sin  nkx 

where  k  =  2lf/X.  An  =  Cn  cos<{)n.  and  Bn  =  -Cn  sin^n-  Equation  5  in  more  compact  form  is  given  by 
00  “ 

flxl  =  A()  +  ]T  Ancosnkx+  2J  Bn  sin  nkx 
n  =  1  n  =  I 

Equation  7  is  called  the  trigonometric  Fourier  series. 

The  process  of  determining  the  values  of  the  coefficients  A„,  An,  and  Bn  is  called  Fourier  analysis.  They  are 
determined  from 
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s.-4-f* 

X  J  (i 


flxldx 


±  f  x 

An~x  J  0 


and 


fix)  cos  nkx  dx 
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X 


flxl  sin  nkx  dx 


where  A„  is  the  average  value  of  flxl.  the  zero  frequency  component.  The  magnitude.  Mn,  of  the  coefficients  A„  and 
Bn.  is  given  by 


Mn  =  VAn2  +  Bn-’ 

and  their  phase  angle.  $n,  the  relative  position  or  phase  shift  between  the  coefficients  An  and  Bn.  is  given  by 

.  ,  Bn 

*en  =  tan  "» 

An 


I6| 


1 101 


It) 


Mn  and<}>n  provide  a  description  of  f(xl  in  the  frequency  domain  of  the  function  f(xl,  called  a  magnitude  and  phase 
spectrum ,  in  contrast  to  fix)  which  is  a  description  in  the  space  domain.  Magnitude  and  phase  will  be  discussed  further 
in  the  following  sections. 

2.2  Complex  Fourier  Series 

Integration  can  be  computed  with  ease  if  the  real  sinusoids  are  expressed  in  terms  of  complex  exponentials,  exp‘Jknx_ 
where  J  =V-1.  The  coefficients  of  the  exponential  Fourier  series  are  found  from 


fix)  exp-Jknx  dx 


and  since  a.„=  a  Jj( where  *  denotes  the  complex  conjugate) 

fix)—  £  anexpJknx 
n  =  -oo 

2.3  The  Fourier  Transform  Pair 


111) 


112) 


As  the  period  X  is  increased,  more  and  more  of  the  function  fix)  will  be  included  in  the  expression.  As  X"*00,  the 
spacing  between  the  harmonics  becomes  differential,  x-»dx,  and  the  number  of  components  becomes  infinite,  n-+°°. 
The  summation  formally  passes  into  an  integral  and  leads  to  the  Fourier  transom  of  fix)  given  by 


fix)  exp-Jkx  dx 


1131 


fix)  can  be  recovered  from 


oo 

•OO 


FI Jk  I  expJkx  dk 


(14) 


which  is  called  the  inverse  Fourier  transform  of  Fix).  The  functions  fix)  and  F(jk)are  called  a  Fourier  transform  pair. 
The  general  practice  is  followed  using  lower  case  letters  for  space  functions  and  capital  letters  for  the  transform. 

2.4  The  One-Dimensional  Discrete  Finite  Fourier  Transform 

The  previous  Fourier  transform  pair  describes  a  continuous  function  fix)  that  lends  itself  to  optical  Fourier  transforms 
le.g.,  O’Neill,  1963,  Lipson  and  Lipson,  1969;  Hecht  and  Zajac,  1974).  However,  because  digital  filtering  techniques 
will  be  used  in  the  Fourier  transform  domain,  the  finite,  discrete  Fourier  transform  is  used  here.  Brigham  1 1974) 
provides  a  lucid  theoretical  development  las  well  as  amplification  of  what  follows)  of  the  continuous  to  the  discrete 
Fourier  transform.  The  modifications  of  going  from  a  continuous  to  a  discrete  representation  are  concerned  with  space 
domain  sampling,  truncation  effects  from  the  limited  number  of  components,  and  frequency  domain  sampling. 

Briefly,  the  continuous  function  fix)  is  sampled  at  discrete  points  by  delta  functions,  <5(x-kXl.  over  the  sampling 
interval  X-  The  sampled  function  fglx)  is  given  by 
00 

fglx)  =  £  f(kXl5lx-kX) 

k  =-oo  115) 

If  equal  interval  sampling  occurs  at  equal  to  or  greater  than  two  samples  for  the  highest  frequency  component 
present  in  fglx)  (called  the  Nyquist  frequency),  then  the  sampled  function  fglx)  will  be  a  good  representation  of  fix). 
However,  if  a  more  coarse  sampling  rate  occurs,  the  result  is  aliasing:  distortion  introduced  into  the  sampled  function 
due  to  spectrum  overlap. 


Neither  the  visual  system  nor  a  digital  computer  can  deal  with  an  infinite  number  of  samples.  Therefore,  the  sampled 
function  f8lxl  is  truncated  by  some  function.  A  rectangular  function  is  given  by 

tlxl=l  0  <  x  <  X  H<>) 

=  0  elsewhere 

where  X  is  the  length  of  the  truncation  function.  The  sampled  truncated  function  now  becomes 
N-l 

fst  (xl  =  E  f(kX)6(x-  kXl  1 171 

k  =  0 

Improfier  truncation,  e.g..  truncating  fix)  at  each  end  of  the  sample  points,  results  in  additional  frequency  components 
in  the  frequency  domain,  called  leakage,  because  each  Nth  point  of  one  period  adds  to  the  first  point  of  the  next  period. 

Such  considerations  lead  to  the  finite  discrete  Fourier  transform  pair. 

.  n  x  N-l 

F(nX  j =  flkXlexp-J2irnk/N  I18al 

wh-'e  n  =  0.  1. 2.  .  .  .  ,  N-l 

1  N_1  /  n  \ 

flk.Xl  =  u  E  F[-=r)  exPJ27r.nk/N  H8bl 

—  n  n =o  \NXy 

where  k  =  0.  1 . 2.  .  .  .  .  N- 1 .  The  discrete  Fourier  transform  is  a  special  case  of  the  continuous  Fourier  transform.  If 
the  samples  of  the  original  function  fix  I  are  one  period  of  a  periodic  function,  then  the  Fourier  transform  of  this  periodic 
function  is  given  by  Fquation  18a. 

It  must  be  stressed  that  the  discrete  Fourier  transform  is  a  complete  transform  in  its  own  right.  Only  under  sjH'cial 
conditions  will  there  be  equivalence  between  the  discrete  and  continuous  Fourier  transforms.  Differences  between  the 
two  transforms  are  due  to  the  sampling  and  truncation  requirements  of  the  discrete  Fourier  transform. 

The  discrete  and  continuous  Fourier  transforms  will  be  exact  I  within  a  scaling  constant)  if: 


111  fix)  is  periodic 
121  fix)  is  band-limited 

1 3 1  the  sampling  rate  is  at  least  two  times  the  highest  frequency  component  of  fix)  and 

141  the  truncation  function  is  non-zero  over  exactly  one  period  tor  an  integer  multiple  period  I  of  fix  I. 

Recall  that  improper  sampling  results  in  aliasing.  Improper  truncation  results  in  leakage.  There  are  techniques  for 
reducing  the  effects  of  aliasing  and  leakage  when  nonperiodic  functions  are  Fourier  transformed.  However,  these 
techniques  are  beyond  the  scope  of  the  present  discussion  and  can  be  found  in  Brigham  1 1*4741. 


2.7)  The  Two-Dimensional  Discrete  Finite  Fourier  Transform 

So  far.  only  one-dimensional  transforms  have  been  discussed.  The  everyday  patterns  that  are  seen  are  two-dimensional 
intensity  distributions  on  the  retina  (excluding  depth  and  time)  and  require  a  two-dimensional  finite,  discrete  Fourier 
transform  for  analysis: 
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(19bl 


p(x,  y 


V 

i-ff  £ 

—  —  u  =-u 


V 
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V  =-v 


•&*>-  [-J2"(f + 1)] 


xu  yv 

where  x  =  2u  +  l,y  =  2v+  i  and  2  IT  ^  ,  2n  -y  are  the  spatial  frequencies  (cycles  per  unit  length  x,  y )  in  the  Fourier 

transform  plane  (u,  vl.  Bracewell  (1956)  and  Brigham  (1974)  provide  good  discussions  of  the  properties  of  the  one-  and 
two-dimensional  transforms. 


Insight  into  the  two-dimensional  Fourier  transform  can  be  gained  by  using  Euler’s  identity  to  separate  the  complex 
exponential  term  into  complex  trigonometric  functions.  Equation  19a  then  becomes 


(201 


The  arguments  of  the  cosine  and  sine  terms  determine  the  spatial  frequency  and  the  orientation  of  the  two-dimensional 
sinusoids.  The  spatial  frequency,  f,  and  orientation  of  the  sinusoids.  0.  is  given  by 


(21) 


0  =  tan-l 


u 

v 


The  cosine  term  is  the  even  symmetric,  real  (Re)  part  of  the  Fourier  transform;  the  sine  term  is  the  odd  symmetric, 
imaginary  tlml  part  of  the  Fourier  transform.  As  with  the  trigonometric  series,  the  magnitude  M  and  phase  4> of  the 
Fourier  transform  are  given  by 


and 


Mn=VRen2+Imn2 


tan 


K 

Ren 


2.5. 1  Two-Dimensional  Sinusoids 


(221 


The  two-dimensional  Fourier  transform  can  be  interpreted  as  a  filter  process  by  which  patterns  are  filtered  by  a  bank 
of  two-dimensional  sinusoids  of  different  spatial  frequency  and  orientation.  The  degree  to  which  the  spatial  frequency 
component  is  present  in  the  spatial  structure  of  a  pattern  that  is  "captured"  by  a  particular  sinusoid  is  given  by  the 
magnitude.  The  phase  is  needed  to  determine  where  the  magnitude  belongs  with  respect  to  the  spatial  position  of 
the  pattern. 

These  two-dimensional  sinusoidal  (cosine  and  sine  function  ( filters  are  shown  in  Figure  1 .  Three  vertical,  integrally- 
related  spatial  frequencies — harmonics — are  shown.  Horizontal  and  oblique  harmonics  are  also  needed  to  define  the 
three-harmonic  set  of  filters  for  the  two-dimensional  Fourier  transform.  They  would  be  identical  to  those  shown  but 
at  a  different  orientation.  A  three-dimensional  view  of  these  sinusoids  is  shown  in  Figure  2. 

The  previous  analysis  has  shown  that  any  pattern  can  be  decomposed  into  sinusoids.  Any  system  that  processes 
patterns  can  be  described  completely  by  specifying  the  properties  of  sinusoids,  spatial  frequency  and  orientation  or 
magnitude  and  phase,  as  they  are  processed  by  the  system.  The  reduction  in  magnitude  of  the  sinusoids  by  an  isotropic 
(no  change  in  magnitude  with  orientation)  system  can  be  measured  in  terms  of  contrast  (the  difference  in  luminance 
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between  the  light  and  dark  bars  referenced  to  the  mean  luminance).  The  relationship  which  specifies  the  ratio  of  input 
to  output  contrast  is  the  modulation  transfer  function  [ MTF] .  The  MTF,  then,  specifies  the  ability  of  any  system  to 
process  spatial  contrast  information  about  patterns. 


Now  that  the  theory  of  the  discrete  Fourier  transform  has  been  outlined,  the  next  section  will  provide  a  discussion  of  the 
digital  techniques  that  are  used  to  compute  the  discrete  Fourier  transform  of  objects. 


CHAPTER  3 


DIGITAL  IMAGE  PROCESSING 


The  previous  chapter  provided  some  theory  about  the  two-dimensional  discrete  Fourier  transform.  This  chapter 
presents  a  discussion  of  the  digital  computer  techniques  that  are  used  to  implement  the  theory.  Digital  techniques  are 
used  to  compute  the  two-dimensional  Fourier  transform  of  simple  objects — squares.  Finally,  the  relationship  between 
size  and  bandwidth  will  be  demonstrated  using  digitally  computed  Fourier  transforms. 

Two  special  purpose  digital  image  processing  systems  and  two  general  purpose  digital  computer  systems  were  used  to 
perform  digital  filtering  to  create  the  filtered  images  that  are  shown  in  this  dissertation.  General  information  on  digital 
image  processing  may  be  found  in  Andrews  1 19701. 

3. 1  The  Image  Analysis  Facility 

Most  of  the  digital  filtering  was  done  using  the  Image  Analysis  Facility  (Ginsburg,  Cook,  Mott-Smith,  19721  at  the  Air 
Force  Cambridge  Research  Laboratories  in  the  United  States.  The  Image  Analysis  Facility  (IAF|  used  a  vidicon 
camera  to  scan  complex  grey-level  pictures,  transforming  intensity  levels  into  video  signals.  The  video  was  then  filtered, 
sampled,  and  channeled  to  an  analogue-to-digital  (A/D)  converter  by  a  32-channel  multiplexer.  The  A/D  converter 
digitized  the  sampled  video,  creating  data  acceptable  to  the  digital  computer,  a  PDP-1 .  A  digital  computer  programme 
stored  one  sampled  vertical  line  of  the  picture  video  into  computer  memory  line  by  line.  Each  line  was  stored 
successively  on  a  digital  magnetic  drum.  After  drum  storage,  the  unprocessed  picture  was  transferred  and  stored  on 
magnetic  tape  for  future  processing  or  returned  to  the  computer  for  immediate  processing.  The  picture  was  displayed 
on  a  cathode  ray  tube  (CRT)  for  observation  or  photographing  before  and/or  after  processing. 

Although  the  IAF  was  very  efficient  for  most  picture  processing,  it  was  too  slow  to  be  considered  for  use  as  an 
interactive  processor  for  the  two-dimensional  Fourier  transform.  Therefore,  a  much  faster  digital  computer,  a  CDC 
6600,  was  used  to  transform  and  filter  the  pictures.  Computer  programmes  were  written  to  link  the  data,  which  were 
transferred  by  magnetic  tape  from  one  computer  system  to  the  other.  The  pictures,  filtered  using  the  CDC  6600 
general  purpose  computer,  were  returned  to  the  IAF  for  photographing.  This  method  proved  most  efficient  for  routine 
filtering.  Furthermore,  high  speed  printers  and  plotters,  coupled  to  the  CDC  6600  system,  quickly  produced 
numerical  printouts  and  plots  of  the  pictures,  Fourier  spectra,  and  filtered  images,  as  desired.  The  numerical  printouts 
contain  10  or  64  amplitude  levels  of  the  filtered  images.  The  contour  plots  contain  10  or  16  amplitude  levels  that  have 
been  equally  spaced  between  the  maximum  and  minimum  values  of  the  filtered  images. 

Another  advantage  of  the  CDC  system  was  that  simple  patterns,  such  as  two-tone  alphanumerics.  could  be  coded  onto 
punch  cards  and  read  directly  into  the  CDC  computer  for  processing,  thereby  saving  scanning  time. 

The  two-tone  patterns  were  cardpunched  for  32  by  32  pel  (picture  element)  grids.  The  Fourier  spectra  were  obtained 
from  a  Fast  Fourier  Transform  programme,  e.g.,  see  Brigham  (1974),  available  on  the  CDC  6600  computer.  Spatial 
filtering  was  accomplished  in  the  transform  domain  as  previously  discussed.  The  filtered  spectra  were  inverse  Fourier 
transformed  to  create  the  filtered  images.  Interpolation  programmes  on  the  IAF  were  used  to  enlarge  the  patterns  to 
236  by  236  pels.  Except  for  the  original  patterns,  the  transforms  and  filtered  images  were  extrapolated  using  sine 
sinitx 

x  =  n  x  as  a  general  smoothing  function  to  avoid  coarse  quantization  artifacts  when  they  were  viewed.  Compare, 
for  example,  the  smoothed  and  original  filtered  images  in  Figure  62,c,d.  The  differences  between  the  original  and 
extrapolated  data  are  negligible  when  viewed  at  the  appropriate  viewing  distances  discussed  later. 

The  grey-level  (multi-tone)  patterns  were  digitized  by  the  video  scanner  of  the  IAF  for  128  by  128  pel  grids.  Filtering 
was  done  as  before  on  the  CDC  66(H)  computer.  The  128  by  128  pel  patterns  were  enlarged  to  420  by  448  pels  using 
extrapolation  programmes  on  the  IAF.  However,  no  smoothing  function  was  necessary  for  these  larger  patterns. 

In  general,  it  is  necessary  to  band-limit  objects  that  are  digitally  processed  to  avoid  the  effect  of  aliasing  (Section  2.4). 
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An  object  is  typically  sampled  5  to  10  times  more  than  the  Nyquist  sample  and  then  filtered  to  the  Nyquist 
frequency.  This  technique  insures  that  spatial  frequencies  up  to  and  including  the  Nyquist  frequency  have  been 
sampled  adequately  and  that  the  object  is  adequately  band-limited.  This  technique  was  not  needed  in  the  studies  made 
in  this  dissertation.  The  simple  binary  objects  processed  in  this  dissertation  were  not  created  by  a  digital  sampling 
system,  but  were  defined  analytically  on  digital  grids.  Thus,  those  objects  are  band-limited  to  begin  with.  Further, 
since  the  Fourier  transform  produces  spatial  frequencies  only  up  to  and  including  the  Nyquist  frequency,  there  will  be 
no  aliasing.  However,  the  grey-level  pictures  in  this  dissertation  were  sampled  by  a  digital  scanning  device.  Therefore, 
it  is  possible  that  the  finest  details  of  the  pictures  were  not  adequately  sampled.  However,  any  artifacts  front  this 
sampling  should  be  evident  from  viewing  the  sampled  picture.  Since  there  were  not  any  artifacts  seen  between  the 
original  and  sampled  pictures,  it  was  assumed  that  there  were  no  serious  aliasing  problems.  F urther.  since  those 
objects  were  also  spatial  filtered  in  the  Fourier  transform  domain,  only  spatial  frequencies  through  the  Nyquist 
frequency  were  used  and  no  further  aliasing  effects  occurred.  Finally,  aliasing  maximally  affects  the  higher  spatial 
frequencies.  Since  almost  all  the  objects  are  low  pass  filtered  to  some  degree,  any  possible  results  of  aliasing  are 
greatly  attenuated. 

The  original  objects,  Fourier  spectra,  and  filtered  images  were  photographed  from  the  CRT  of  the  I AF  using 
Polaroid  55  P/N  film  to  create  the  64  grev-level  pictures  shown  in  this  dissertation. 

3.2  The  SEMPER  image  Processing  System 

Some  of  the  digital  filtering  was  done  with  an  image  processing  system  called  SEMPER,  developed  for  the  analysis  of 
electron  microscopy  imagery  in  the  Electron  Microscopy  Branch  of  Applied  Physics  at  the  University  of  Cambridge. 

The  input  scanning  device  for  SEMPER  (a  commercial  system  called  SCANDIG  built  by  Technical  Operations.  Inc.: 
Joyce  Loebl  Ltd.  in  the  U.K.  I  is  a  high-speed  rotating  microdensitometer  which  converts  pictorial  information  into 
digital  form.  It  measures  and  records  optical  densities  in  two  dimensions  through  any  flexible  photographic 
transparency  at  the  rate  of  20,000  readings  per  second. 

Light  from  a  50-watt  halogen  lamp  passed  through  an  iris,  was  deflected  by  a  mirror,  and  focused  onto  the  sample 
mounted  on  the  drum.  The  transmitted  light  was  collected  by  an  objective  lens  and  passed  through  a  variable 
aperture  onto  the  face  of  a  photomultiplier  tube.  The  photomultiplier  tube  converted  the  collected  light  into  an 
electrical  signal  proportional  to  the  amount  of  light  transmitted.  A  log  amplifier  converted  the  electrical  signal  into 
units  of  density  which  were  input  into  an  A/D  converter.  The  A/D  converted  changed  the  signal  into  8-bit  binary- 
data  which  were  then  stored  on  magnetic  tape. 

The  data  on  magnetic  tape  were  then  processed  using  the  SEMPER  system.  The  heart  of  the  SEMPER  system  is  a 
PDP-8/E  digital  computer  with  moving-head  disk  storage.  The  picture  data  were  processed  using  the  SEMPER 
executive  language  commands  under  teletype  control.  Operations  performed  on  the  pictures  were  monitored  on  a 
display  oscilloscope. 

The  digital  data  of  the  processed  pictures  were  used  to  create  photographs  containing  256 grey  levels.  The  Photowrite 
system  IP-1500,  Optronics  International,  Inc.;  Joyce  Loebl.  Ltd.,  in  the  U.K.  I  is  a  high-speed  digital  film-writing 
system  which  consists  of  an  unexposed  film  clamped  to  the  outside  of  a  circular  rotating  drum.  The  light  from  a 
light-emitting  diode  modulated  by  the  digital  data  was  passed  through  an  aperture  and  focused  onto  the  film.  The 
drum,  film,  and  optical  path  were  enclosed  in  a  light  tight  enclosure  which  is  removed  to  develop  the  film. 

Kodak  high-speed  Infrared  film  (4143  Thick;  10.2  by  12.7  cm  I  was  used  to  create  negatives.  The  exposed  film  was 
developed  in  Ilford  PW  Universal  developer  (1:4,  78°F)  for  2.5  minutes,  given  a  normal  bath  in  water,  then  “fixed" 
for  2.5  minutes  using  Super  Amfix  (1:31.  Prints  were  made  from  the  developed  film  using  Ilfospeed  Grade  O  0.1  M 
paper  exposed  at  F5.6  for  1  to  1.5  seconds.  The  linearity  of  the  developing  and  printing  processes  was  determined  from 
step  wedges  placed  along  the  sides  of  the  pictures. 

Although  simple  band-pass  spatial  filtering  could  be  done  using  the  SEMPER  system,  practical  considerations,  such  as 
available  storage  space  and  time  required  for  computation,  made  it  necessary  to  use  the  central  computer  system,  an 


IBM  370.  for  much  of  the  picture  processing.  Therefore,  as  with  previous  picture  processing,  the  spatial  filter  design 
and  much  of  the  spatial  filtering  were  accomplished  on  a  large  computer  system  and  the  SEMPER  system  was  used  as 
an  input/ output  device  to  digitize  and  create  pictures  of  the  filtered  objects. 

3.3  A  Demonstration:  Two-Dimensional  Fourier  Magnitude  Spectra  of  Squares 

The  two-dimensional  Fourier  transforms  of  two  different  size  squares  were  computed,  using  the  previously  described 
techniques,  to  create  the  magnitude  spectra  shown  in  Figure  3. 

Both  the  magnitude  and  phase  components  of  the  transform  are  required  to  describe  a  pattern  uniquely.  The 
magnitude  components  are  the  amplitudes  associated  with  each  spatial  frequency  and  orientation  in  the  frequency 
domain.  The  phase  components  represent  the  position  and  orientation  of  the  associated  spatial  frequencies  in  the 
original  pattern  space.  Since  the  phase  of  the  transform  has  not  been  modified  for  the  results  presented  in  this 
dissertation,  the  phase  is  not  shown. 

The  two  spatial  coordinates,  x  and  y,  of  the  original  squares  correspond  to  two  spatial  frequency  coordinates  in  the 
transform  domain,  u  and  v.  These  spectra  have  been  arranged  like  the  diffraction  patterns  that  would  be  observed  in  a 
device  performing  an  optical  Fourier  transform,  e.g.,  Lipson  and  Lipson  ( 1%9I.  The  spectra  are  bilaterally 
symmetrical  across  any  axis  that  goes  through  the  centre  of  the  transform.  The  central  spectral  component  at  the 
origin  lfu  =  0 .  fv  =  01  is  the  zero  frequency  or  average  intensity  value  of  the  pattern.  The  spatial  frequencies  go  from 
low  to  high,  radially  outward  from  the  centre  of  the  spectrum.  The  central  high-intensity  region  is  the  central 
maximum  which  contains  about  00  percent  of  the  pattern  energy . 

3.3. 1  The  Relationship  Between  Space  and  Spatial  Frequency 

An  important  point  about  a  linear  transform,  such  as  the  Fourier  transform,  is  that  no  information  is  gained  or  lost  in 
transformation.  However,  the  spatial  information  about  an  object  is  rearranged,  as  shown  by  the  magnitude  spectra 
of  Figure  3.  In  more  complex  patterns,  the  large  parts  of  the  pattern  will  be  found  in  the  low  spatial  frequencies  and  the 
small  parts  will  be  found  in  the  high  spatial  frequencies.  This  rearrangement  of  spatial  information  in  the  frequency 
domain  makes  the  filtering  of  spatial  information  such  as  size  and  orientation  of  complex  patterns  quite  simple,  as  will 
be  shown  in  the  next  chapter. 


CHAPTER  4 


SPATIAL  FILTERING 

In  this  chapter  the  Fourier  transform  is  put  to  work  to  spatially  filter  patterns.  First,  the  difference  between  using  the 
Fourier  transform  rather  than  convolution  techniques  to  filter  patterns  is  discussed.  Demonstration  of  spatial  filtering 
of  a  complex  scene  will  follow. 

4.1  Convolution  Versus  Fourier  Transformation 

There  are  two  general  techniques  by  which  spatial  filtering  can  be  accomplished.  A  pattern  can  be  spatially  filtered  by 
convolution,  that  is,  multiplying  the  pattern  point  by  point  with  a  filter  function  that  is  translated  over  the  entire 
pattern  and  adding  the  results,  e.g.,  Bracewell  ( 1965).  An  alternative  method  is  to,  first,  transform  (change)  the 
pattern  from  the  space  domain  into  a  spatial  frequency  domain,  e.g.,  Fourier  transformation,  then  directly  attenuate 
(that  is.  reduce  in  amplitude)  or  extract  certain  unattenuated  spatial  frequency  information  and,  finally,  perform  an 
inverse  transform  to  reconstruct  the  pattern  from  that  attenuated  or  extracted  information.  These  techniques  can 
produce  identical  results.  Indeed,  there  is  a  theorem  that  states  that  the  Fourier  transform  of  the  convolution  of  two 
functions  is  equal  to  the  product  of  the  Fourier  transforms  of  the  functions  taken  separately,  e.g.,  Bracewell  ( 1965). 

Although  convolution  can  be  used  to  spatially  filter  a  pattern,  it  does  not  provide  explicit  spatial  frequency  and  phase 
information  about  the  pattern.  Unlike  transform  methods  of  spatial  filtering,  convolution  is  an  implicit  filtering 
operation.  In  other  words,  only  limited  insight  into  the  filtering  of  patterns  is  gained  using  convolution. 

Considerable  insight  can  be  gained,  however,  by  noting  how  the  Fourier  transform  filters  spatial  information. 

Putting  Equation  20  into  words:  The  two-dimensional  Fourier  transform  yields  spatial  frequency /orientation 
information  by  multiplying  the  pattern  point  by  point  with  two-dimensional  arrays  of  sinusoids  (e.g..  Figure  21  of 
different  spatial  frequency  and  orientation,  and  then  summing  over  all  the  points  for  each  sinusoidal  array.  Thus, 
although  the  computations  are  different,  results  of  a  Fourier  transformation  can  be  viewed  as  similar  to  convolution: 
and,  because  sinusoids  are  symmetric  patterns,  it  can  also  be  looked  upon  as  a  cross-correlation  of  the  input  pattern 
with  two-dimensional  sinusoidal  arrays  over  a  range  of  spatial  frequencies  and  orientations.  Fourier  transformation 
differs  from  convolution  in  two  ways:  It  does  not  require  that  one  of  the  patterns  be  reversed,  and  it  does  not  translate 
or  move  one  pattern  across  the  other.  The  Fourier  transform  does  its  filtering  with  functions  that  remain  fixed  in 
position.  This  important  difference  between  transform  and  convolution  techniques  should  be  kept  in  mind  when  either 
technique  is  suggested  as  a  model  of  visual  processing,  as  will  be  discussed  in  Section  19.9.3. 

4.2  Spatial  Filtering  in  the  Fourier  Transform  Domain 
4.2. 1  One-Dimensional  Ideal  Band-Pass  Filters 

Spatial  filtering  in  the  Fourier  transform  domain  is  done  by  multiplying  the  spectral  components  of  the  pattern  by  some 
filter  function.  Mathematically,  a  simple  one-dimensional  band-pass  filter  is  given  by 

Hhplu)=  I  for  some  -uiU  S.  u  (23) 

—  0  elsewhere 

An  ideal  low-pass  filter  simply  multiplies  some  central  portion  of  the  magnitude  spectrum  by  one  and  the  rest  of  the 
transform  by  zero.  The  inverse  process  results  in  high-pass  filtering.  An  ideal  band-pass  filter  is  made  by  multiplying 
some  finite  region  away  from  the  centre  of  the  magnitude  by  one  and  the  rest  of  the  transform  by  zero. 

Examples  of  one-dimensional  ideal  low-,  high-,  and  band-pass  filters  are  shown  in  Figure  4, a,  1 ,2,3.  The  equivalent 
line-spread  functions  (the  Fourier  transform  of  the  filters  above)  that  would  produce  similar  filtering  by  convolution 
are  shown  in  Figure  4,b.  1 ,2,3.  The  general  effect  of  each  filter  on  a  square  wave  (the  square-wave  response)  is  shown 
in  Figure  4,e,  1 ,2.3.  The  specific  shapes  of  the  filtered  squares  will  depend  on  the  specific  band  widths  and  shapes  of 
the  filters. 
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The  filtered  spectrum  can  be  synthesized  back  into  the  space  domain  by  an  inverse  Fourier  transform,  P"  ^ ,  which 
results  in  a  filtered  object  called  an  image.  This  method  enables  the  observer  to  determine  what  spatial  information 
about  the  original  object  was  contained  in  certain  parts  of  the  spectrum.  The  filtered  image,  Pf(xl,  is  given  by 

Pf(x!  =  P-l(ul{[P(u)Hbplu)]]  (241 

4.2.2  Two-Dimensional  Band-Pass  Spatial  Filters 

The  properties  of  the  one-dimensional  filters  discussed  in  the  previous  section  can  be  extended  to  two-dimensions,  as 
shown  in  Figure  5,a,b,c.  Other  general  shapes  and  configurations  of  two-dimensional  filters  are  shown  in  Figure 
5,d,e,f.  The  shaded  areas  shown  in  these  figures  are  the  regions  of  spatial  frequencies  that  are  passed  by  the  filter. 

These  spatial  filters  will  be  used  to  select  various  bands  of  spatial  frequency  and  orientation  of  many  different  patterns 
in  subsequent  sections. 

A  three-dimensional  view  of  two-dimensional  ideal  low-pass  circular  and  square  filters  and  their  respective 
point-spread  functions  is  shown  in  Figure6,a,l,b,l,a,2,b,2.  The  relationship  between  these  filters  and  single 
channels  in  the  visual  system  is  discussed  in  Section  8.2.1. 

4.2.3  Spatial  Filtering  of  a  Complex  Scene 

This  section  demonstrates  spatial  filtering  in  the  two-dimensional  discrete  Fourier  domain  using  a  complex  visual 
scene — part  of  the  Bridge  of  Sighs  at  St.  John’s  College,  Cambridge.  This  scene  was  chosen  because  of  its  rich  periodic 
structure  in  both  the  horizontal  and  vertical  orientations. 

The  digitized  scene  and  its  magnitude  spectrum  are  shown  in  Figure  7, a,  1 ,2.  Figure  7, a, 3  shows  the  magnitude 
spectrum  that  was  sectioned,  normalized,  and  recombined  to  show  the  high  frequencies.  The  repetitive  structure  along 
the  vertical  and  horizontal  orientations  in  the  magnitude  spectra  generally  reflects  the  periodicity  in  the  scene. 

First,  the  high  spatial  frequencies  were  removed  (Figure  7,b).  (The  diagonal  lines  shown  in  the  pictures  of  the  spectra 
in  Figure  7,d-h  represent  the  region  of  spatial  frequencies  that  are  passed  by  the  spatial  filter. )  Only  the  general  form  of 
the  bridge  remains  when  the  scene  contains  only  12  and  18  low  spatial  frequencies.  Figure  7,b,c. 

Figure  7,d,e  shows  the  result  of  passing  only  the  high  spatial  frequencies— those  greater  than  12  and  18.  Removing  the 
low  spatial  frequencies  greatly  reduces  the  contrast  over  broad  spatial  regions.  The  details  of  the  scene  now 
predominate. 

,  The  previous  filtering  was  isotropic;  the  same  bandwidth  was  used  at  all  orientations.  Next,  a  predominant  range  of 
horizontal  components  was  passed  (Figure  7,f  I.  Since  the  horizontal  spatial  frequencies  result  from  vertical  objects  in 
the  scene,  it  is  not  surprising  to  see  generally  vertical  objects  that  reflect  the  overall  vertical  periodicity  in  the  original 
scene.  Note  the  vertical  bars  in  the  portals  of  the  bridge.  Filtering  most  of  the  vertical  components  produced  broad 
horizontal  objects,  as  shown  in  Figure  7,g.  Filtering  the  left  oblique  parts  of  the  spectrum  produces  generally  right 
oblique  parts  of  the  scene  (Figure  7,h). 

The  periodicity  of  the  windows  can  be  detected  on  the  horizontal  axis  of  the  spectrum  shown  by  the  circle  in  Figure  7,i. 
This  small  amount  of  information  was  filtered  out  to  create  Figure  7,i,  where  the  repeated  light  and  dark  bars  are 
plainly  seen.  This  repeated  structure  appears  sinusoidal  because  only  relatively  few  components  of  the  spectrum  have 
been  passed.  Since  each  point  in  the  spectrum  represents  the  filtering  of  the  original  pattern  with  one  sinusoid,  it  is 
reasonable  to  expect  that  filtering  out  that  component  results  in  the  sinusoidal  appearance. 

Such  demonstrations  show  the  power  of  spatial  filtering  in  two  dimensions.  Form  and  detail  of  different  size  and 
orientation  can  be  extracted  in  a  global  manner  quite  easily  from  even  the  most  complex  scene. 
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4.3  Summary  of  Fourier  Analysis  and  Spatial  Filtering 

The  previous  chapters  have  shown  sinusoids  to  be  powerful  tools  in  both  the  theoretical  and  practical  aspects  of 
describing  patterns.  Their  use  in  Fourier  analysis  resulted  in  two-dimensional  spectra  that  were  used  as  a  means  of 
spatially  filtering  a  complex  scene.  The  next  chapter  provides  the  experimental  data  that  are  needed  to  make  spatial 
filters  to  probe  the  filtering  characteristics  of  the  visual  system  using  the  preceding  methods. 
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CHAPTER  5 


THE  OVERALL  SPATIAL  FILTERING  CHARACTERISTICS 
OF  THE  HUMAN  VISUAL  SYSTEM 


i).  I  Some  Basics:  Spatial  Frequency,  Contrast,  and  Contrast  Sensitivity 

Recall  that  a  sine-wave  grating  is  a  repeated  sequence  of  light  and  dark  bars  whose  luminance  profile  varies 
sinusoidally  about  a  mean  luminance  with  distance  (Figures  1,2).  Here  two  variables  of  a  sine-wave  grating  are 
considered:  spatial  frequency  and  contrast.  The  width  of  one  light  and  one  dark  bar  of  a  grating  is  one  cycle,  or  the 
period  of  the  grating.  The  reciprocal  of  the  period  is  the  spatial  frequency.  Spatial  frequency  is  expressed  by  the 
number  of  cycles  of  the  grating  that  occur  over  a  particular  distance.  The  spatial  frequency  of  an  object  can  be 
expressed  by  cycles  per  object  (c/o)  dimension  or,  more  commonly,  by  cycles  per  unit  of  visual  angle.  The  number  of 
cycles  per  object  dimension  is  determined  by  the  size  of  the  particular  dimension  of  some  part  of  or  the  entire  object  and 
is  independent  of  viewing  distance.  Cycles  per  unit  of  visual  angle,  more  commonly  called  cycles  per  degree  (c/d),  is 
determined  by  the  viewing  distance. 

The  luminance  difference  of  the  light  and  dark  bars  determines  the  contrast  of  the  grating.  The  definition  of  contrast 
most  often  used  is  that  first  given  by  Michelson  ( 1*1271: 

„  Umax  —  Uni  in 
Umax  "h  Uni  in 

where  Lmax  and  Lmj„  are  the  maximum  and  minimum  liuninanceof  the  bars  of  the  grating. 

Figure  8  shows  a  sine-wave  grating  whose  contrast  is  constant  but  whose  spatial  frequency  is  changing  logarithmically. 

( Figures  8.  *).  and  10  are  unpublished  photographs  supplied  by  F.  W.  Campbell  and  J.  G.  Robson.  I  The  wider  bars 
at  the  left  are  low  spatial  frequencies  when  compared  to  the  narrow  bars  on  the  right  which  are  high  spatial  frequencies. 

The  effect  of  contrast  on  a  grating  of  a  fixed  spatial  frequency  is  shown  in  Figure  0.  There  is  a  square-wave  grating  on 
the  left  side  and  a  sine-wave  grating  on  the  right.  The  medium  contrast  of  the  grating  at  the  bottom  is  reduced 
logarithmically  towards  the  top  of  the  grating.  Note  near  the  top  that  the  contrast  of  the  grating  is  so  low  that  only  a 
uniform  grey  area  is  seen.  The  length  of  the  visible  grating  will  change  with  viewing  distance. 

If  the  contrast  of  a  grating  is  increased  from  below  visibility  to  where  the  grating  is  just  seen,  then  the  grating  is  said 
to  have  reached  threshold  contrast.  Gratings  of  different  spatial  frequencies  require  different  amounts  of  contrast  to 
reach  threshold  for  the  observer.  The  reciprocal  of  the  threshold  contrast  is  called  contrast  sensitivity.  If  the  log  spatial 
frequency  grating  is  shown  with  log  contrast,  as  in  Figure  10,  the  limit  of  visible  grating  results  in  the  inverted 
U-shaped  function  found  by  contrast  sensitivity  measurements. 

The  importance  of  contrast  in  enabling  one  to  see  various  spatial  information  in  objects  cannot  be  stressed  too  much. 
The  loss  of  spatial  information  as  contrast  is  reduced  is  demonstrated  by  a  portrait  shown  in  Figure  1 1 .  Various  details 
about  the  portrait.  such  as  the  hair  over  the  forehead,  are  selectively  lost  as  the  contrast  is  reduced  from  1(H)  jiercent  to 
<>  jiercent.  At  the  lowest  contrast.  6  (icrcent.  only  a  ghost-like  face  remains. 

:>.2  The  Contrast  Sensitivity  Function 

There  must  be  a  limit  in  any  real  system  to  the  range  of  information  it  can  process.  Just  as  the  retina  is  sensitive  to 
the  limited  range  of  electromagnetic  energy  that  is  called  light,  the  eye  is  also  sensitive  to  a  limited  range  of  size  of 
objects.  Microscopes  and  telescopes  were  invented  to  extend  the  range  of  object  sizes  that  can  be  seen.  The  contrast 
sensitivity  function  describes  the  range  of  object  sizes  that  can  be  passed  by  a  visual  system. 

The  range  of  object  sizes  that  can  be  seen  became  a  major  concern  with  the  advent  of  telecommunications.  It  is  not  too 
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surprising  that  Schade  ( 19561,  an  electrical  engineer  faced  with  the  problem  of  how  much  spatial  information  to 
transmit  for  television,  made  the  first  measurements  of  the  spatial  bandwidth  of  the  visual  system.  He  found  that  the 
human  visual  system  responded  best  to  a  limited  range  of  spatial  frequencies. 

The  importance  of  such  measurements  was  realized  and  techniques  improved  by  Campbell  and  Robson  1 1964,  1%8I. 
Figure  12  shows  a  typical  contrast  sensitivity  function  that  they  obtained  with  sine-wave  gratings.  Note  that  the  visual 
system  is  more  sensitive  at  about  1  to  5  c/d  and  less  sensitive  at  higher  and  lower  spatial  frequencies. 

Carlson,  Cohen,  and  Gorog  ( 1977 1  used  one-  and  two-dimensional  sine-wave  gratings  to  measure  contrast  sensitivity. 
The  contrast  sensitivity  obtained  with  two-dimensional  gratings  agrees  well  with  the  sensitivity  obtained  with 
one-dimensional  gratings  from  0.2  to  20  c/d  (mean  luminances  of  0.034  to  34  millilamberts).  Thus,  the  contrast 
sensitivity  function  can  be  viewed  as  the  "window”  through  which  spatial  objects  are  viewed  under  threshold 
conditions. 

5.2. 1  Factors  Which  Affect  the  Shape  of  the  Contrast  Sensitivity  Function 

The  general  shape  of  the  contrast  sensitivity  function  is  affected  by  the  grating  waveform,  luminance,  focus,  pupil  size, 
orientation,  the  number  of  bars  of  the  grating,  which  part  of  the  retina  is  tested,  and  whether  the  gratings  are  stationary 
or  change  in  phase  by  either  flicker  or  movement.  For  example,  Campbell  and  Robson  (19681  showed  that  the 
low-spatial-frequency  loss  that  increases  with  decreasing  spatial  frequency  below  about  1  c/d,  when  measured  with 
sine-wave  gratings,  becomes  relatively  less  when  square-wave  gratings  are  used  to  measure  contrast  sensitivity. 

Decreasing  the  mean  retinal  luminance  causes  the  peak  sensitivity  and  the  high-frequency  cutoff  of  the  contrast 
sensitivity  to  shift  to  lower  spatial  frequencies,  e.g,,  Patel  (1966);  Van  Nes,  Koenderink,  Nas,  and  Bouman  (19671.  The 
low-frequency  falloff  disappears  at  scotopic  luminances,  and  the  visual  system  acts  as  a  low-pass  filter  rather  than  a 
band-pass  filter. 

The  effect  of  focus  and  pupil  size  on  contrast  sensitivity  was  studied  by  Campbell  and  Green  (19651.  In  general,  they 
found  that  the  sensitivity  to  high  spatial  frequencies  decreased  as  defocus  increased.  Optimum  contrast  sensitivity, 
which  agreed  with  a  diffraction  limited  system,  was  obtained  with  a  2  mm  diameter  pupil.  Increased  pupil  size  caused 
a  shift  in  performance  from  that  of  a  diffraction  limited  optical  system. 

The  contrast  sensitivity  to  gratings  changes  as  a  function  of  orientation  (e.g.,  Campbell  and  Kulikowski,  1966; 
Mitchell,  Freeman,  and  Westheimer,  1967).  In  general,  contrast  sensitivity  is  greatest  for  vertical  gratings,  slightly 
less  for  horizontal,  and  least  for  oblique  gratings,  especially  at  the  high  spatial  frequencies. 

It  is  important  to  consider  how  many  bars  make  a  grating.  Estevez  and  Cavonius  (1976)  showed  that  fewer  than  about 
eight  bars  caused  a  decrease  in  the  low-frequency  falloff  of  the  contrast  sensitivity  function.  Furthermore,  by  using 
fewer  bars,  they  showed  that  edge  effects  and  surround  luminance  can  affect  contrast  sensitivity.  For  example, 
sine-wave  gratings  that  ended  in  sine  phase  and  had  a  dark  surround  resulted  in  reduced  contrast  sensitivity  when 
compared  to  similar  gratings  having  a  uniform  light  surround. 

It  is  important  to  note  that  moving  gratings  (Robson,  1966)  and  flickering  gratings  (Kulikowski  and  Tolhurst,  19731 
result  in  a  substantial  decrease  in  the  low-frequency  falloff.  These  effects  are  important  because  the  retinal  image  of  an 
object  is  never  perfectly  stationary  under  normal  viewing  conditions. 

There  is  less  sensitivity  to  high  spatial  frequency  gratings  seen  in  the  periphery  than  in  the  fovea  (Daitch  and  Green, 
1969;  Sharpe  and  Tolhurst,  1973).  The  reduction  in  peripheral  sensitivity  is  not  due  to  the  optics  (Daitch  and  Green, 
19691.  Hilz  and  Cavonius  ( 1974)  showed  that  sensitivity  to  gratings  decreased  by  a  factor  of  10  from  the  central  retina 
to  16  degrees  in  the  periphery. 

As  Schade  ( 1956)  pointed  out,  a  linear  optical  system  cannot  selectively  reduce  low  spatial  frequencies  to  a  greater 
degree  than  high  spatial  frequencies.  The  observed  low-frequency  falloff  must  therefore  represent  the  filtering 
properties  of  neurological  mechanisms  in  the  visual  system. 
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The  filter  characteristics  for  the  optics  and  the  retinal-brain  system  have  been  separated  in  the  elegant  ex|>eriments  of 
Campbell  and  Green  ( 1063)  and  Campbell  and  Gubisch  ( 1966).  The  contrast  sensitivity  to  gratings  is  not  limited  by  the 
optics  of  the  eye  under  normal  viewing  conditions.  Thus,  the  high  frequency  cutoff  also  represents  the  filtering  by 
neurological  mechanisms  in  the  visual  system. 

5.:$  Single-  Versus  Multi-channel  Visual  Models 

A  channel  is  defined  as  one  or  more  neural  mechanisms  that  respond  best  to  some  region  along  a  stimulus  dimension 
and  less  or  not  at  all  to  other  regions  of  that  same  dimension,  e.g..  spatial  frequency  Isizel  and  orientation. 

Does  the  overall  contrast  sensitivity  reflect  the  cascaded  result  of  optics  and  neurons  that  can  be  treated  as  a  single 
spatial  filter?  This  single-channel  view  has  been  used  by  some  researchers,  e.g..  Hatliff  I  166.>l,  Patel  1 1 I.  Cornsweet 
(  1970),  to  account  for  certain  observations — for  example,  Mach  bands.  However,  it  predicts  the  largest  Mach  bands 
with  square  waves,  not  ramps,  e.g..  Cornsweet  I  1970),  which  does  not  agree  with  observation  (McCullough.  I9.>,d. 
Furthermore,  such  a  model  cannot  account  for  other  observations,  such  as  the  independent  detection  of  spatial 
frequency  components  of  patterns  at  threshold,  as  is  discussed  in  later  sections. 

An  alternative  to  the  single-channel  model  is  the  multi-channel  model.  The  multi-channel  model,  first  proposed  by 
Campbell  and  Robson  ( 1968),  suggests  that  the  visual  system  contains  a  number  of  channels  (filtering  mechanisms  I. 
each  selective  to  a  limited  range  of  spatial  frequencies.  They  showed  that  the  ratio  of  the  contrast  thresholds  for  a 
sine-wave  versus  a  square-wave  grating  of  the  same  spatial  frequency  was  4/71,  a  v  alue  predicted  front  the  amplitude  of 
the  fundamental  Fourier  components  of  the  two  gratings.  The  important  asjtect  of  this  result  was  that  the  value  4/71  was 
more  or  less  invariant  for  gratings  of  spatial  frequencies  greater  than  about  1  c/d.  A  single  filter  model  would  have 
predicted  a  greater  sine-to-square  threshold  ratio  for  low  and  medium  frequencies  because  the  higher  harmonics  of  the 
square  waves  could  contribute  to  the  output  of  the  filter.  The  ratio  of  4/77  would  be  predicted  only  when  the  third 
harmonic  (3fl  of  the  square  wave  was  outside  the  range  of  the  filter,  e.g..  when  3f  was  greater  than  about  30  to  60  c/d. 
These  findings  strongly  demonstrated  linearity  of  detection  at  threshold  and  led  Campbell  and  Robson  to  suggest  that 
there  are  a  number  of  channels  in  the  human  visual  system,  each  tuned  to  a  different  range  of  spatial  frequencies.  I  he 
overall  contrast  sensitivity  function,  then,  represents  the  envelope  of  the  combined  activity  of  all  of  the  individual 
channels. 

As  was  shown  in  Section  .3.3. 1 ,  size  and  spatial  frequency  are  inversely  related  to  one  another.  Thus,  any  research  done 
in  terms  of  size  can  be  reinterpreted  in  terms  of  spatial  frequency.  Certain  researchers  who  have  preferred  to  work  in 
the  space  domain  have  found  results  similar  to  those  researchers  working  in  the  spatial  frequency  domain.  Kv  idenee  for 
independent  size  mechanisms  was  provided  by  Pantle  and  Sekuler  ( 1968).  They  found  that  the  detection  of  gratings 
was  affected  most  by  adapting  (see  Section  6.1.11  to  square-wave  gratings  of  similar  spatial  frequencies.  Thomas  ( 16701 
discussed  the  need  for  size-tuned  mechanisms  to  account  for  the  detection  of  certain  objects. 

3.4  The  Filtering  Characteristics  of  Suprathreshold  Vision 

The  shape  of  the  contrast  sensitivity  function  is  based  upon  the  threshold  detection  of  gratings.  <  >ther  techniques,  such 
as  contrast  matching,  provide  a  family  of  curves  that  relate  the  overall  filter  characteristics  of  the  v  isual  sy  stem  to 
suprathreshold  levels  of  contrast.  Although  initially  investigated  by  Davidson  I  I ‘>681  and  Watanabe.  Mori.  Nagata. 
and  Hiwatashi  1 1968),  the  most  thorough  investigation  of  contrast  matching  has  been  done  by  Georgeson  and  .Sullivan 
( 16731.  They  investigated  contrast  matching  using  sine-wave  gratings  in  the  peripheral  retina,  as  well  as  central  v  ision, 
and  in  different  orientations. 

F’igure  1.3  shows  results  of  contrast  matching  by  the  author  (see  Section  A-2..3  for  methodsl.  This  family  of  curves  shows 
that  the  familiar  contrast  sensitivity  curve,  found  at  low  contrasts  (up|>ermost  curve),  smooths  into  almost  linear 
horizontal  curves  (lower  curves)  at  high  contrast. 

These  kinds  of  results  prompted  Georgeson  and  Sullivan  ( 1673)  to  postulate  that  the  constant  valued  sensitiv  ity  seen  at 
high  contrast  was  due  to  independent  spatial  frequency  channels.  Each  channel  reached  peak  sensitivity  due  to 
individual  gain  mechanisms  sensitive  to  contrast. 
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5.5  Summary 


This  chapter  has  presented  the  teachniques  and  examples  that  provide  information  about  the  overall  filtering 
characteristics  of  the  visual  system  from  threshold  to  suprathreshold.  Further  discussion  introduced  the  concept  of 
channels  in  the  visual  system.  The  bandwidth  and  shape  of  these  channels  are  discussed  in  the  next  -■  ctions. 

Many  pages  could  easily  be  devoted  to  all  the  psychophysical  experiments  that  provide  evidence  for  the  existence  of 
mechanisms  in  the  human  visual  system  tuned  to  different  ranges  of  spatial  frequency  (size)  and  orientation,  i.e., 
channels.  Readers  interested  in  the  details  of  that  evidence  are  referred  to  several  recent  reviews  (Campbell,  1974; 
Sekuler,  1974;  Robson,  1975;  Breitmeyer  and  Ganz,  1976:  Braddick,  Campbell,  and  Atkinson,  1977;  and  Graham, 
1977).  Also,  readers  interested  in  the  phase  specificity  of  channels,  difference  between  low  and  high  spatial  frequency 
channels,  suprathreshold  effects  such  as  the  spatial  frequency  shift,  and  interactions  among  channels  and  edge  and  bar 
channels  are  referred  to  the  same  reviews.  Since  the  main  concern  here  is  with  the  filtering  characteristics  of  channels, 
only  those  papers  that  provide  bandwidth  and  orientation  measurements  about  channels  are  discussed  in  certain 
following  sections. 


CHAPTER  6 


THE  SPATIAL  FILTERING  PROPERTIES  OF  CHANNELS 
DETERMINED  FROM  PSYCHOPHYSICAL  DATA 


6. 1  The  Measurement  of  the  Spatial  Bandwidth  of  Channels 

Several  psychophysical  techniques  have  been  used  to  determine  the  spatial  frequency  and  orientation  selectivity  of 
channels  in  the  human  visual  system.  They  include  selective  adaptation,  masking,  threshold  summation,  and  visual 
evoked  responses.  Selected  results  of  each  will  be  discussed. 

6.1.1  Selective  Adaptation 

Selective  adaptation  is  determined  from  the  reduction  in  sensitivity  to  one  stimulus  due  to  prolonged  exposure  to 
another. 

The  first  information  about  the  bandwidth  of  channels  came  from  Blakemore  and  Campbell  (19691.  They  found 
selective  threshold  elevation  (reduced  contrast  sensitivity  I  after  adaptation  to  sine-wave  gratings  of  different  spatial 
frequencies  from  3  to  20  c/d.  Their  data  suggested  that  the  bandwidth  of  the  channels,  measured  by  the  half-height  of 
the  threshold  elevation  of  the  test  grating,  was  about  1.2  octaves.  Blakemore,  Muncey,  and  Ridley  (19731  replicated  the 
adaptation  method  with  similar  techniques;  they  found  a  bandwidth  of  1 .3  octaves. 

Lange,  Sigel.  and  Stecher  (1973 (extended  the  findings  of  narrow  bandwidth  channels  by  Sachs.  Nachmias,  and 
Robson  ( 19711  (see  Section  6. 1 .31.  They  had  gubjects  adapt  to  single  components  of  stimuli  that  consisted  of  two 
sine-wave  gratings.  They  found  that  the  narrow  bandwidth  within  which  the  two  gratings  were  not  detected 
independently  remained  the  same  even  after  threshold  elevation  of  the  component  gratings.  Their  results  were  used  to 
estimate  the  channel  bandwidth  to  less  than  J  octave. 

6.1.2  Masking 

Another  technique  for  measuring  channel  bandwidth  is  to  simultaneously  present  a  masking  and  test  signal.  The 
bandwidth  of  the  channel  is  measured  by  the  reduced  effect  of  the  masking  signal  as  its  frequency  is  shifted  from 
the  test  frequency. 

Carter  and  Henning  (19711  used  one-dimensional,  broad-band  filtered  noise  as  a  masking  stimulus  for  gratings.  They 
found  that  the  most  effective  masking  was  achieved  by  noise  whose  bandwidth  was  1  to  2  octaves  around  the  mean  of 
the  test  frequency. 

Stromeyer  and  Julesz  ( 19721  also  used  band-limited  noise  to  mask  gratings.  They  found  that  threshold  elevation 
decreased  by  one-half  when  the  frequency  of  the  test  grating  was  about  0.6  m’tave  away  from  the  band-limited  noise. 
However,  after  increasing  the  width  of  the  band-limited  noise,  they  found  that  the  most  effective  masking  centered  over 
the  test  frequency  occurred  within  a  noise  bandwidth  of  2  wtaves. 

6.1.3  Threshold  Summation 

The  threshold  summation  technique  determines  the  contrast  threshold  of  a  stimulus  measured  alone  and  when  it  is 
superimposed  on  another  subthreshold  stimulus.  The  sensitivity  of  the  mechanism  that  detects  the  original  stimulus  is 
measured  by  the  reduction  in  threshold  produced  by  the  presence  of  the  second  stimulus. 

Graham  and  Nachmias  ( 1971 1  used  threshold  summation  to  measure  contrast  thresholds  for  the  detection  of  different 
phase  combinations  of  two  sine-wave  gratings  having  a  spatial  frequency  ratio  of  1  to  3.  Since  the  peak-to-trough 
amplitude  was  quit*  different  for  each  phase  condition,  if  single  waveforms  were  detected  by  different  channels,  then 
different  thresholds  should  have  occurred.  However,  independent  thresholds  were  found  whether  the  gratings  were  in 
peaks-add  or  peaks-subtract  phase.  In  addition  to  providing  good  evidence  for  a  multi-channel  model,  their  results  also 
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sug8est  that  the  channels  are  not  affected  by  gratings  whose  frequencies  are  separated  by  a  factor  of  3. 

Very  narrow  bandwidth  measurements  were  made  by  Sachs,  et  al.  11971 )  using  threshold  summation.  They  used  a 
complex  analysis  designed  to  distinguish  between  the  detection  of  complex  gratings  by  a  single-  and  a  multi-channel 
model.  In  support  of  the  multi-channel  model,  they  found  independent  detection  of  the  two  components  of  a  complex 
grating  with  frequency  relationships  f  |  /f  ■>  =  4/5  or  5/4.  The  resulting  channel  bandwidth  was  only  about  0.4  octave 
at  a  centre  frequency  of  14  c/d.  Narrow  bandwidths  were  also  found  with  similar  techniques  by  Quick  and  Reichert 
119751. 

An  elegant  attempt  to  provide  the  two-dimensional  structure  of  a  single  channel  was  made  by  Mostafavi  and  Sakrison 
1 19761.  Their  model  represent  a  channel  as  an  incoherent  detector  (one  which  sums  similar  frequency  components 
from  different  parts  of  the  visual  field  independent  of  phase)  consisting  of  a  band-pass  filter,  followed  by  a 
transformation  and  spatial  summation.  From  the  detection  of  pseudo-random  images  (band-pass  filtered  Gaussian 
white  noise),  they  found  a  bandwidth  of  about  1  octave  around  the  centre  frequency  and  an  angular  bandwidth  of 
20  degrees. 

6. 1 .4  Visual  Evoked  Response 

Campbell  and  Maffei  ( 1970)  used  measurements  of  the  visual  evoked  response  ( VER )  to  determine  the  spatial  tuning 
of  channels.  The  VER  is  an  electrical  potential,  recorded  from  electrodes  placed  on  the  scalp,  which  is  produced  by 
responses  to  visual  stimuli.  A  signal  is  obtained  by  averaging  the  voltage  sensed  by  the  electrodes  over  a  large  number  of 
trials.  The  averaging  process  is  needed  to  cancel  out  the  random  noise  in  the  signal.  The  stimuli  were  sine-wave  gratings 
that  were  alternated  in  phase  at  a  rate  of  8  cycles/second.  A  linear  relationship  was  found  between  the  logarithm  of  the 
stimulus  contrast  and  the  amplitude  of  the  evoked  response.  The  slope  of  the  graph  of  log  contrast  versus  amplitude 
was  used  to  measure  the  channel  bandwidth.  Two  gratings  of  different  spatial  frequencies  were  presented,  one  in  the 
upper  half  of  the  field  and  the  other  in  the  lower.  The  slope  of  the  curve  increased  as  the  difference  between  the 
spatial  frequency  of  the  two  gratings  increased.  The  maximum  slope  was  found  when  the  difference  between  the  spatial 
frequency  of  the  two  gratings  was  1  octave  or  more.  They  concluded  that  channels  separated  by  1  octave  are 
independent. 

6.2  The  Measurement  of  the  Orientation  Selectivity  of  Channels 

6.2. 1  Selective  Adaptation 

Blakemore  and  Nachmias  ( 1971 )  presented  a  high  contrast  grating  at  different  orientations  to  adapt 
orientation-selective  mechanisms  in  central  vision.  The  orientation  tuning  characteristics  were  measured  from  the 
threshold  elevation  of  a  sine-wave  grating  of  the  same  frequency  as  the  adapting  grating.  Threshold  elevation  decreased 
as  the  difference  between  the  orientation  of  the  adapting  grating  and  test  grating  increased.  Threshold  elevation 
decreased  to  half  of  maximum  when  the  orientation  difference  was  ±  7  degrees. 

Movshon  and  Blakemore  ( 1973),  using  the  same  adaptation  technique,  found  similar  orientation  selectivity,  +7.5 
degrees  averaged  from  two  subjects.  Furthermore,  they  showed  that  the  orientation  tuning  did  not  vary  with  the  spatial 
frequency  of  the  gratings. 

'Hie  orientation  selectivity  in  peripheral  vision  was  measured  by  Sharpe  and  Tolhurst  1 1973).  Using  the  same 
adaptation  technique  as  Blakemore  and  Nachmias  ( 1971 1,  they  found  that  the  orientation  tuning  was  somewhat 
broader.  12  to  20  degrees. 

6.2.2  Masking 

Campbell  and  Kulikowski  (19661  measured  the  orientation  selectivity  of  the  visual  system  with  the  technique  of 
masking.  A  masking  grating  varying  in  orientation  was  superimposed  on  a  test  grating  of  the  same  spatial  frequency. 
Maximum  masking  effect,  reduced  contrast  sensitivity,  occurred  when  the  two  gratings  were  of  the  same  orientation 
and  was  reduced  to  half  when  the  difference  in  orientation  between  the  two  gratings  was  12  degrees  for  vertical  test 
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gratings  and  15  degrees  for  oblique  test  gratings. 

6.2.3  Threshold  Summation 

Kulikowski,  Abadi,  and  King-Smith  (19731  used  threshold  summation  to  measure  the  orientation  selectivity  of 
gratings.  They  found  quite  narrow  tuning  characteristics.  Threshold  elevation  was  decreased  to  half  at  about  ±  3 
degrees  and  virtually  disappeared  at  about  +  18  degrees. 

6.2.4  Monocular  Rivalry 

Campbell  and  Howell  (1972)  reported  what  they  termed  monocular  rivalry  with  sine-wave  gratings.  When  two 
sine-wave  gratings  are  superimposed  perpendicularly  to  one  another,  they  appear  to  alternate  — only  one  being  visible 
at  a  time.  One  grating  fades  in  while  the  other  fades  out.  Atkinson,  Campbell,  Fiorentini,  and  Maffei  (1973 1  found 
that  the  rate  of  alternation  can  appear  to  be  quite  rapid  under  certain  conditions.  Campbell,  Gilinsky.  Howell,  Riggs, 
and  Atkinson  ( 19731  measured  the  rate  of  alternation  as  the  orientation  of  the  two  gratings  was  changed.  They  found 
that  the  gratings  alternated  only  when  the  orientation  difference  between  them  was  greater  than  15  degrees. 

It  is  quite  difficult  to  explain  such  tuning  without  the  concept  of  independent  channels.  The  i  15  degrees  found  to 
produce  monocular  rivalry  is  similar  to  the  bandwidth  of  orientation  mechanisms  found  using  other  psychophysical 
techniques.  Since  orientation-selective  neurons  have  been  found  only  in  the  cortex,  some  aspects  of  monocular  rivalry- 
are  presumably  a  function  of  the  tuning  properties  of  cortical  mechanisms.  It  may  be  that  monocular  rivalry  due  to 
crossed  gratings  occurs  when  two  different  groups  of  orientation-selective  mechanisms  are  stimulated . 

6.2.5  Visual  Evoked  Response 

Campbell  and  Maffei  ( 19701  used  the  visual  evoked  response  to  measure  the  orientation  selectivity  to  gratings.  After 
adaptation  to  a  high  contrast  grating,  the  subject  observed  low  contrast  gratings  and  the  evoked  potential  was  recorded. 
This  adaptation  resulted  in  a  reduction  of  the  amplitude  of  the  evoked  response.  However,  the  reduction  found  when 
the  two  gratings  were  at  the  same  orientation  virtually  disappeared  when  the  orientation  difference  between  the 
gratings  was  20  degrees. 

This  chapter  has  presented  some  of  the  literature  that  provides  psychophysical  and  electrophysiological  evidence  for 
mechanisms  in  the  visual  system  that  are  selective  for  spatial  frequency  (size)  and  orientation.  These  results 
reflect  the  overall  response  of  the  visual  system  to  grating  stimuli.  Do  they  reflect  the  measurements  of  individual 
mechanisms  in  the  visual  system?  These  measurements  are  discussed  in  certain  following  sections. 
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CHAPTER  7 


THE  SPATIAL  FILTERING  PROPERTIES  OF  CHANNELS 
DETERMINED  FROM  NEUROPHYSIOLOGICAL  DATA 

The  notion  that  the  overall  envelope  of  the  filter  characteristics  of  the  visual  system  results  from  the  activity  of 
quasi- independent  spatial  frequency  channels  would  seem  to  follow  naturally  from  the  single  cell  recordings  from 
animals  by  many  early  researchers,  e.g.,  Kuffler  ( 19531.  That  notion  has  not  always  been  accepted,  as  was  discussed  in 
Section  5.3. 

7. 1  The  Receptive  Field 

Kuffler  1 19531  showed  that  the  receptive  fields  (the  visual  space  to  which  neurons  respond)  in  the  retina  of  the  cat  were 
of  two  types:  "on-centre  and  "off-centre. "  The  firing  rate  of  an  "on-centre"  neuron  increased  if  a  small  spot  of  light, 
placed  in  the  centre  of  the  receptive  field,  was  turned  on.  Some  cells  that  have  a  maintained  discharge  in  the  absence 
of  stimulation  increase  their  discharge  rate  temporarily  when  a  small  spot  of  light,  placed  in  the  centre  of  the  receptive 
field,  is  turned  off.  These  are  "off-centre"  neurons.  Small  spots  of  light  turned  on  in  the  surround  of  “on-centre" 
neurons  or  turned  off  for  the  "off-centre”  neurons  reduced  their  characteristic  response.  This  effect  is  called  lateral 
inhibition. 

7.2  Feature  Detectors 

Many  researchers  have  interpreted  receptive  fields  from  the  pioneering  research  of  Kuffler  (1953)  and  Hubei  and 
Wiesel  ( 19591  as  feature  detectors.  Barlow,  Narashimhan,  and  Rosenfeld  ( 1972)  provide  an  extensive  review  of 
literature  related  to  feature  detectors. 

Feature  detectors  are  composed  of  receptive  fields  that  are  uniquely  selective  to  disks,  edges,  etc.;  they  respond 
maximally  when  a  perfect  match  between  the  receptive  field  and  a  stimulus  with  a  particular  geometric  shape  occurs. 

In  other  words,  a  disk  of  light  is  signaled  because  a  receptive  field  of  the  correct  size  and  position  with  respect  to  the 
visual  field  has  been  excited. 

An  alternative  view  of  receptive  fields  has  developed  over  the  last  10  years.  It  removes  the  criterion  of  feature  detection 
from  the  receptive  field  and  replaces  it  with  a  far  more  useful  concept,  that  of  a  channel.  The  substitution  of  the  word 
channel  for  feature  detector  is  no  mere  play  on  words.  Within  the  concept  of  channel  lies  the  recognition  that  complex 
visual  objects  can  be  analyzed  with  a  far  more  general  range  of  spatial  parameters  than  that  considered  by  advocates 
of  feature  detectors.  This  is  not  to  say  that  there  is  no  use  for  some  of  the  properties  of  feature  detectors  such  as  edge 
detectors.  Such  concepts  can  be  quite  useful,  but  not  to  the  exclusion  of  other  conceptual  mechanisms.  The  existence 
of  channels,  with  their  concomitant  selection  of  spatial  frequency  (size),  orientation,  and  phase,  and  the  importance  of 
contrast,  allows  the  concept  of  spatial  filter  theory  for  perception  Co  be  investigated  more  fully. 

*  **  i  1 1  i  n 

7.3  Evidence  for  Spatial  Filters  in  the  Mammalian  Visual  System 

I  his  section  briefly  samples  the  research  that  underlies  the  concept  of  neurophysiological  channels  and  provides  the 
basic  parameters  for  describing  spatial  filters.  Robson  ( 1975)  provides  a  more  extensive  review. 

Enroth-Cugell  and  Robson  ( 196hl  provided  the  first  investigations  of  the  filter  characteristics  of  single  cells  using 
sine-wave  gratings.  They  found  that  the  retinal  ganglion  cells  of  the  cat  are  optimally  tuned  to  particular  ranges  of 
spatial  frequency.  Each  cell  has  its  own  contrast  sensitivity  function.  The  spatial  frequency  that  produces  the 
maximum  response  varies  from  cell  to  cell. 

The  most  important  finding  of  that  research  was  not  the  individual  tuning  characteristics  of  the  retinal  ganglion  cells 
but  that,  in  general,  there  are  two  types  of  cells:  X  and  Y.  The  more  narrowly  tuned  X-cells,  which  predominate  in 
the  central  retina,  provide  linear  spatial  summation.  The  relatively  broader  tuned  Y-cells  are  nonlinear.  This  evidence 
for  linear  processing  by  one  class  of  cells  in  the  mammalian  retina  suggests  that  the  linear  transmission  of  spatial 
information  of  objects  is  important  for  subsequent  visual  processing. 
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Maffei  and  Fiorentini  (1973)  also  investigated  the  tuning  characteristics  of  retinal  ganglion  cells.  Robson  (19751 
interpreted  their  results  as  showing  that  the  response  amplitude  of  the  cells  was  proportional  to  the  contrast  of  gratings 
at  low  contrast  levels;  but  at  higher  levels  of  contrast,  a  logarithmic  relationship  was  found.  Although  some  behaviour 
of  the  retinal  ganglia  is  not  completely  linear,  Robson  (1975)  still  argued  that  contrast  rather  than  absolute  luminance 
variation  is  the  more  appropriate  stimulus  variable  to  measure. 

The  tuning  characteristics  of  the  retinal  ganglion  cells  are  relatively  broad  when  compared  to  those  of  the  cells  found  at 
the  lateral  geniculate  nucleus  (LGN)  and  the  striate  cortex  (Maffei  and  Fiorentini,  1973).  Furthermore,  the 
predominantly  isotropic  (circularly  symmetric)  receptive  fields  found  in  the  retina  and  LGN  give  way  to  anisotropic 
receptive  fields  in  the  striate  cortex  (Hubei  and  Wiesel  1959,  1962,  1968).  The  rectangularly  shaped  receptive  fields  in 
the  striate  cortex  occur  in  a  variety  of  sizes  and  orientations  and  have  different  numbers  of  excitatory  and  inhibitory 
zones.  It  is  not  surprising  that  such  cells  respond  maximally  to  gratings  whose  orientation  and  spatial  frequency  best 
match  the  orientation  and  width  of  the  areas  that  form  the  receptive  field.  The  close  relationship  between  the  spatial 
weighting  of  cortical  cells  and  sinusoidal  gratings  is  seen  in  Figure  2.  The  maxima  and  minima  of  the  sinusoidal 
gratings  can  be  related  t,  the  excitation  (  +  I  and  the  inhibition  (  -  I  shown  by  the  rectangular  plots  to  the  upper  right 
corner  of  each  grating. 

The  first  tuning  characteristics  of  cortical  cells  in  the  cat  were  obtained  by  Campbell,  Cooper,  and  Enroth-Cugell 
( 1969).  They  found  that  the  optimum  contrast  sensitivity  was  orientation-specific  and  varied  from  cell  to  cell  with 
response  falling  off  above  and  below  the  optimum  spatial  frequency.  Further  grating  studies  on  cortical  cells  in  the  cat 
were  done  by  Glezer,  Ivanoff,  and  Tscherbach  (1973),  and  by  Maffei  and  Fiorentini  (1973).  They  found  that  the 
bandwidth  of  simple  cells  was  about  2  octaves;  that  of  complex  cells  was  somewhat  larger.  Glezer  et  al.  (1973)  and 
Pollen  and  Ronner  ( 1975)  found  that  complex  and  hyper-complex  cells  were  optimally  stimulated  by  periodic  rather 
than  single  stripes. 

Simple  cells  appear  to  be  excited  by  X-cells  and  complex  cells  by  Y-cells  (Stone  and  Dreher,  1973).  Simple  cells  are 
also  more  position-sensitive  to  the  receptive  field  than  are  complex  cells.  These  findings,  coupled  with  the  narrow 
tuning  of  the  simple  cells,  suggest  that  it  is  perhaps  more  appropriate  to  consider  the  behaviour  of  simple  cells  than 
complex  cells  as  linear  spatial  filters.  In  general,  these  studies  show  optimum  tuning  in  simple  cells  of  about  ±15 
degrees  in  orientation  and  about  2  octaves  in  spatial  frequency. 

Recent  investigations  by  DeValois  ( 1976)  of  the  tuning  properties  of  simple  cells  in  monkey  found  very  narrow  tuning 
in  both  orientation  and  spatial  frequency.  Bandwidths  at  half-amplitude  varied  from  0.7  to  greater  than  2  octaves. 
Orientation  tuning  varied  from  ±  4  degrees  to  greater  than  +  50  degrees.  About  25  percent  of  the  cells  were  tuned  to 
only  0.7  to  1 .2  octaves  in  spatial  frequency  and  ±  4  to  ±10  degrees  in  orientation. 

In  addition  to  the  very  narrow  filtering  characteristics  of  cortical  cells,  an  interesting  picture  of  the  organization  of 
spatial  frequency  and  orientation  columns  in  the  cortex  has  been  provided  by  neurophysiologists.  Hubei  and  Wiesel 
(19,4)  found  a  systematic  change  of  orientation  selectivity  in  neurons  from  measurements  made  perpendicularly 
through  a  column  in  the  visual  cortex.  This  systematic  orientation  tuning  was  confirmed  by  Maffei  and  Fiorentini 
( 19771,  who  found  that  tangential  penetrations  through  a  column  resulted  in  systematic  changes  in  the  spatial 
frequency  selectivity  of  the  cells. 

These  data  provide  the  exciting  view  of  the  striate  cortex  composed  of  columns  of  “two-dimensional  machines"  (Hubei 
and  Wiesel,  1974)  that  act  as  a  two-dimensional  array  of  sieves  systematically  tuned  for  orientation  and  spatial 
frequency  and  are  repeated  throughout  the  striate  cortex  like  a  lattice  structure  of  the  finest  crystal. 

7.4  Differences  Between  Bandwidths  of  Channels 

The  bandwidth  estimates  determined  from  psychophysical  measurements  of  Sachs  et  al.  (1971 ),  Lange  et  al.  (19731. 
and  Quick  and  Reichert  ( 1975)  are  quite  narrow,  about  0.5  octave.  Graham  (1975,  1977)  and  Stromeyer  and  Klein 
(1975)  argued  that  considerations  of  probability  summation— detection  occurring  from  a  pooled,  weighted  response  of 
independent  channels — suggested  that  the  bandwidth  of  these  narrow  channels  may  be  underestimated. 
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If  the  bandwidth  of  channels  determined  by  threshold  summation  is  underestimated,  then  what  about  the  veracity  of 
bandwidth  estimates  as  determined  by  masking  and  adaptation?  Pantle  11974)  derived  relative  bandwidths  from 
different  types  of  psychophysical  data.  He  showed  that  the  narrowest  bandwidth  resulted  from  threshold  summation, 
a  broader  bandwidth  resulted  from  selective  adaptation,  and  masking  produced  the  greatest  bandwidth. 

Dealy  and  Tolhurst  ( 1974)  showed  that  contrast  sensitivity  is  reduced  with  gratings  of  any  contrast  above  threshold. 
Their  results  suggested  that  adaptation  may  be  due  to  inhibition  from  which  Braddick,  et  al.  ( 1977 ),  argued  that  the 
range  of  spatial  frequencies  that  can  adapt  a  channel  is  wider  than  that  which  can  stimulate  it  for  detection.  This 
suggests  that  adaptation  and  masking  experiments  provide  an  overestimate  of  the  channel  bandwidth. 

Although  different  bandwidths  appear  to  arise  from  different  psychophysical  methods,  there  is  at  least  one  other  factor 
to  consider.  It  is  also  possible  that  these  different  bandwidths  are  the  result  of  the  channels  being  at  different  levels  of 
stimulation  or  because  the  channels  being  stimulated  have  a  wide  range  of  different  bandwidths.  Since  DeValois  (1976) 
showed  that  there  are  quite  narrowly  tuned  receptive  fields  in  monkey  visual  cortex,  it  seems  likely  that  the  human 
visual  system  also  contains  receptive  fields  of  similar  narrow  bandwidth. 

What  is  surprising  is  not  that  different  psychophysical  methods  yield  different  bandwidths,  but  that  they  generally 
agree  within  1  octave.  Such  good  agreement  suggests  an  important  general  bandwidth  for  filtered  images.  However, 
it  should  he  kept  in  mind  that  a  complex  object  that  extends  over  the  central  retina  is  most  likely  being  filtered  with 
narrow-band  as  well  as  medium-  and  broad-band  mechanisms  due  to  the  different  sizes  of  receptive  fields  that  exist 
across  the  retina. 

In  summary,  many  psychophysical  and  neurophysiological  studies  have  found  mechanisms  whose  spatial  bandwidths 
averaged  1  to  2  octaves.  Orientation  bandwidths  average  ±  1 5  degrees.  The  next  sections  will  present  spatial  filters 
based  on  the  overall  filtering  characteristics  of  the  visual  system  and  the  bandwidth  and  shape  of  channels  that  have 
been  discussed.  ~ 


CHAPTER  8 


CREATING  TWO-DIMENSIONAL  SPATIAL 
FILTERS  FROM  BIOLOGICAL  DATA 


The  previous  chapters  presented  some  of  the  psychophysical  and  neurophysiological  data  about  the  spatial  filtering 
properties  of  the  mammalian  visual  system.  Here  those  data  are  used  to  create  spatial  filters  with  which  to  filter  objects. 

8. 1  Two-Dimensional  Spatial  Filters  Based  on  Contrast  Sensitivity  Data 

in  Section  5.2  it  was  argued  that  the  visual  system  can  process  spatial  information  about  objects  only  through  a  window 
that  can  be  described  by  the  contrast  sensitivity  and  contrast  matching  functions.  Furthermore,  it  was  shown  that  the 
visual  system  is  anisotropic;  that  is,  sensitivity  to  detection  of  objects  varies  as  a  function  of  orientation.  Both  spatial 
frequency  and  orientation  sensitivities  depend  upon  the  contrast  of  the  object,  e.g.,  Georgeson  and  Sullivan  I  19751. 

A  two-dimensional  spatial  filter  can  simulate  the  effects  of  such  filtering  properties  on  complex  objects.  A  two- 
dimensional  filter  allows  the  determination  of  how  much  spatial  information  about  an  object  is  passed  through  the 
early  stages  of  the  visual  system  and  of  the  consequences  for  spatial  information  contained  in  objects  at 
different  orientations. 

There  is  a  continuum  of  filter  functions  that  changes  from  threshold  to  suprathreshold  (see  Section  5.21.  The  limiting 
case  for  spatial  information  is  the  contrast  sensitivity  function.  Therefore,  a  two-dimensional  filter  was  created  based 
on  contrast  sensitivity  data  (Ginsburg.  19731.  Since  objects  will  be  filtered  using  a  static  filter  as  a  modulation  transfer 
function,  the  two-dimensional  contrast  sensitivity  filter  will  be  called  an  MTF. 

Two  general  assumptions  are  made  in  using  the  contrast  sensitivity  function  as  an  MTF.  f  irst,  as  mentioned  in  Section 
5.2,  the  visual  system  does  not  have  one  MTF  but,  rather,  a  family  of  MTFs  that  range  from  threshold  to 
suprathreshold  and  change  as  a  function  of  the  variables  discussed  in  Section  5.2. 1 .  Thus,  the  particular  conditions 
under  which  the  contrast  sensitivity  function  was  obtained  must  be  specified  in  order  for  it  to  be  used  as  an  MTF. 
Second,  MTFs  based  on  contrast  sensitivity  measurements  obtained  at  threshold  imply  that  the  response  (output  I  of 
the  system  is  constant  and  that  it  changes  as  a  function  of  the  amplitude  and  frequency  of  the  input  sinusoids;  in  other 
words,  the  system  is  linear  (see  Section  1.21.  This  is  perhaps  the  most  stringent  assumption  that  has  to  be  made.  Hay 
and  Chesters  ( 19721  argued  that  signal-transfer  functions  cannot  be  deduced  solely  from  threshold  measurements 
because  those  measurements  depend  greatly  on  system  noise.  Since  they  found  that  noise  vai  i  d  as  a  function  of  retinal 
distance,  the  visual  system  is  nonlinear  at  threshold.  Henning,  Hertz,  and  Broadbent  (19751  found  that  gratings  with 
three  sinusoidal  components  of  high  spatial  frequency  interacted  with  a  sinusoidal  grating  two  octaves  lower  in 
frequency.  That  finding  suggests  nonlinear  distortion  of  high,  but  not  low,  spatial  frequencies.  Although  it  must  la* 
accepted  that  any  real  system  has  some  degree  of  nonlinearity,  the  important  point  is  the  degree  of  nonlinearity  that 
can  be  assumed  for  the  overall  operation  of  the  system.  If  the  visual  system  were  not  linear  to  a  first  approximation. 
Campbell  and  Robson  ( l'M)8l  and  Campbell,  Carjienter,  and  Levinson  ( 1969)  could  not  have  predicted  the  visibility  of 
complex  waveforms  from  the  response  of  the  visual  system  to  sine  waves.  Thus,  it  is  highly  unlikely  that  the 
nonlinearity  findings  by  researchers  such  as  Hay  and  Chesters  should  refute  the  use  of  the  contrast  sensitivity  function 
as  an  MTF.  It  is  in  this  spirit  of  first  order  approximation  of  linearity  that  a  two-dimensional  MTF  was  constructed 
from  contrast  sensitivity  data.  Unless  this  linear  systems  approach  is  taken,  the  relative  importance  of  the 
nonlinearities  of  the  visual  system  will  be  difficult  to  determine. 

Since  the  single  most  complete  set  of  contrast  sensitivity  data  then  available  for  spatial  frequency  and  orientation  was 
that  of  Campbell,  Kulikowski,  and  Levinson  ( |9M»),  their  data  were  used  to  calculate  the  parameters  for  the  MTF 
filter.  The  contrast  sensitivity  values  for  spatial  frequencies  from  1  to  32  c/d  for  sine-wave  gratings  presented  at  0,  45, 
and  'XI  degrees  of  rotation  were  obtained  from  Figures  2  and  5  of  their  re|M>rt.  Lines  were  drawn  by  eye  through  the 
data  points  of  the  three  subjects  and  averaged  for  the  spatial  frequencies  shown  in  Apj>endix  B.  The  contrast 
sensitivities  were  normalized  to  the  maximum  value  (213.3  at  h  c/d  at  0  degrees  orientation).  The  normalized  functions 
are  plotted  with  solid  lines  in  Figure  14.  Values  of  percent  contrast  sensitivity  at  integer  spatial  frequencies  were 
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placed  on  a  rectangular  grid.  The  off-axis  values  between  0,  45,  and  90  degrees  were  interpolated  by  eye,  especially 
noting  the  spatial  frequency  values  of  Figures  1  and  4  of  Campbell,  et  al.  (19661.  These  values  were  arranged  in 
digitized  circles  that  approximated  a  polar  representation  for  the  rectangular  grid  that  was  used  for  the  digital  filtering 
(Figure  15,a). 

Note  that  only  one  quadrant  of  the  two-dimensional  MTF  was  created.  The  grating  data  collected  at  135  degrees  were 
not  used  because  they  differed  little  from  the  data  at  45  degrees,  and  two-fold  symmetry  was  desired  as  a  first 
approximation  for  the  filter.  The  single  quadrant  values  were  replicated  in  the  other  three  quadrants  to  provide  the 
desired  symmetry.  This  filter  will  be  called  MTF(L). 

To  investigate  the  effects  of  increased  contrast  sensitivity  at  the  lower  spatial  frequencies,  1  to  5  c/d,  found  in  other 
studies,  e.g.,  Blakemore  and  Campbell  ( 1969),  a  different  set  of  values  based  upon  their  data  for  the  low  spatial 
frequencies  was  used.  These  are  shown  in  Figure  15,b.  This  filter  will  be  called  MTF(H  I.  The  magnitude  spectra  of 
the  Fourier  transformed  objects  were  multiplied  by  the  values  of  the  MTF  filters  and  then  inverse  Fourier  transformed. 
The  effects  of  the  MTF  spatial  filters  were  determined  from  numerical  printouts,  contour  plots,  and  photographs  of  the 
filtered  images  (see  Section  3.1). 

The  fundamental  frequencies  of  the  patterns  were  attenuated  by  the  contrast  sensitivity  corresponding  to  2  c/d  for  the 
32  by  32  pel  objects  and  0.5  c/d  for  the  128  by  128  pel  objects.  Therefore,  the  MTF  filtered  images  should  be  viewed 
at  about  0.5  degree  (about  120  times  the  width  of  the  object  I  for  the  32  by  32  pel  objects  and  about  2  degrees  (about 
30  times  the  width  of  the  object)  for  the  128  by  128  pel  objects  in  order  to  compare  them  with  the  original  objects  and 
to  determine  the  fidelity  of  the  digital  filtering  to  the  filtering  characteristics  of  the  reader's  own  visual  system. 

The  contrast  sensitivity  for  both  spatial  frequency  and  orientation  can  be  seen  most  easily  from  the  pictures  of  the  MTF 
filters  shown  in  Figure  16.  Figure  16,a  shows  the  MTF(L).  Figure  16. b  shows  the  MTFIH ).  Figure  16, c  and  d  are  due  to 
shorter  exposures  of  the  MTF(H)  to  show  the  details  of  the  filter  characteristics  at  higher  spatial  frequencies.  The  lighter 
and  darker  regions  correspond  to  higher  and  lower  contrast  sensitivities.  Note  also  the  differences  between  different 
spatial  frequencies  and  orientations.  Figure  16, e  shows  a  three-dimensional  view  of  MTFIH ).  Note  the  familiar  shape  of 
the  contrast  sensitivity  function  on  each  side  of  the  x-axis. 


It  should  be  clear  from  this  MTF  filter  that  objects  of  the  same  size  and  intensity  but  at  different  orientations  will  be 
attenuated  differently.  Vertical  objects  will  be  attenuated  less  than  objects  oriented  horizontally  or  at  45  or  135  degrees. 
Since  the  greatest  contrast  sensitivity  occurs  at  3  to  6  c/d,  medium  size  objects  will  be  attenuated  less  than  smaller  or 
larger  objects.  Very  fine  details  about  an  object  (high  spatial  frequencies)  will  be  attenuated  more  by  the  filter  than  will 
intermediate  spatial  frequencies. 

An  alternative  to  these  digital  methods  is  the  use  of  pictures  of  the  MTF  filters  to  create  spatial  filters  for  an  optical 
Fourier  transform  device.  The  negative  of  these  filters,  suitably  reduced  by  microphotographic  techniques,  could  be 
placed  in  the  Fourier  transform  plane  to  produce  filtered  objects  similar  to  those  produced  using  digital  techniques. 

One  caution  in  using  this  optical  technique  is  the  possibility  of  nonuniform  phase  due  to  variations  in  the  thickness  of 
the  material  used  for  the  negative. 

8.2  Two-Dimensional  Spatial  Filters  Based  on  Biological  Channels 

Several  attempts  using  psychophysical  techniques  have  been  made  to  determine  the  shape  of  channels  in  thp  human 
visual  system,  e.g.,  Blakemore  and  Campbell  1 1969);  Stromeyer  and  Klein  11975);  Mostafavi  and  Sakrison  (19761; 
I^gge  1 1977).  However,  each  used  different  methods  and  each  found  a  line-  or  point-spread  function  that  was 
somewhat  different.  The  present  approach  is  to  use  spatial  filters  that  have  the  general  filtering  characteristics  of 
channels:  ideal  low-pass  and  band-pass  filters.  By  changing  the  shape  and  bandwidth  of  these  filters,  a  range  of 
different  point-spread  functions  may  be  obtained,  some  closely  approximating  those  based  on  data.  The  effect  of  these 
changes  in  the  filter  shapes  can  be  seen  in  the  filtered  images.  Previous  spatial  filtering  by  Ginsburg  ( 1971  .a. b;  19731 
and  Ginsburg,  et  al.  ( 1976),  suggested  that  such  changes  had  little  effect  on  the  ability  to  classify  similar  complex 
objects  as  being  similar.  It  was  found  that  a  general  bandwidth  of  low  spatial  frequencies  was  the  most  important 
variable  for  the  classification  of  objects.  Therefore,  the  majority  of  the  spatial  filtering  presented  here  was  done  using 
ideal  low-pass  and  band-pass  fillers.  In  Section  14.3.  a  direct  comparison  will  be  made  on  a  large  set  of  data  between 


ideal  band-pass  and  psychophysically  derived  channel  band-pass  filters  to  justify  this  basic  approach. 

Do  low-pass  digital  filters  correspond  to  mechanisms  in  the  visual  system?  Recall  that  the  channels  in  the  visual  system 
may  be  thought  of  as  discrete  spatial  filters  with  finite  bandw idths.  They  are  not  ideal  low-pass  or  high-pass  optical 
filters.  They  are  band-pass  filters  whose  bandwidth  is  a  function  of  the  size  of  receptive  fields. 

One  point  that  should  be  understood  is  that  the  discrete  finite  Fourier  transform  that  is  implemented  by  using  a  Fast 
Fourier  Transform  (FFTI  is  also  band-limited.  Only  a  finite  number  of  spatial  harmonics  can  be  obtained  from  any 
object.  If  the  size  of  the  object  is  the  same  as  the  sampling  grid,  then  an  integer  multiple  number  of  harmonics  that 
range  from  I  cycle  per  object  to  the  Nyquist  frequency  is  determined.  For  example,  an  object  sampled  on  a  32  by  32 
|>el  grid  yields  1  to  1(>  spatial  harmonics  in  each  dimension.  Therefore,  high-  and  low-pass  filters  in  the  discrete  Fourier 
transform  domain  are  band-pass  filters  with  respect  to  an  optical  Fourier  transform  which  would  contain  higher  and 
lower  spatial  frequencies. 

A  closer  look  is  taken  at  the  line  weighting  function  of  the  one-dimensional  ideal  low-pass  filter  shown  in  Figure  4.b.  I . 
This  space  filter,  convolved  with  an  object,  will  produce  the  same  filtered  image  as  if  the  spectrum  of  the  object  were 
multiplied  by  the  ideal  band-pass  filter  and  inverse  Fourier  transformed. 

But  is  the  rippling  nature  of  this  space  filter  like  that  of  mechanisms  in  the  visual  system?  Mathematically,  the  space 
filter  is  a  sine  function,  given  by 
sin  IT  x 

si,ux  =  T7- 

with  pro|>erties  such  that  sine  (01  =  1,  sine  In)  =  Oforn  =  1,2.3.  .  .  .  .  n  (Bracewell.  19651. 

ft  is  interesting  to  note  the  different  names  attached  to  the  same  parts  of  the  sine  function,  depending  upon  the 
discipline  that  described  them.  In  optics,  the  central  peak  is  called  the  central  maximum  and  the  subsidiary  peaks  and 
valley  s,  the  maxima  and  minima  or  side  lobes.  The  neurophysiologist  and  psychophysicist  describe  spatially  tuned 
mechanisms  having  weighting  characteristics  similar  to  those  given  by  the  sine  function  as  excitatory  (the  central 
maximuml  and  inhibitory  and  disinhibitory  flanks  (minimum  and  maximum  side  lobesl.  The  ideal  low-pass  filter  is 
sometimes  called  a  gate  function  in  electrical  engineering  because  of  its  selective  filtering  properties;  i.e.,  it  "gates" 
certain  signals  through  some  communication  channel. 

8.2. 1  (.omparison  Between  an  Ideal  Low-Pass  Filter  and  Filters  Obtained  front  Biological  Data 

The  sine  function  and  normalized  line-spread  functions  based  on  neurophysiological  and  psychophysical  data  are 
shown  in  Figure  17.  The  solid  line  is  from  table  values  of  the  sine  function  found  in  Bracewell  ( 1%5|.  The  filled  circles 
are  data  obtained  from  the  on-off  response  of  a  flashing  light  moved  through  the  receptive  field  of  a  simple  cell  in  the 
visual  cortex  of  a  cat.  front  Tolhurst  and  Movshon  1 19761.  The  line-spread  function  is  similar  to  that  obtained  bv 
Bishop.  Henry,  and  Smith  1 1971 1  using  different  techniques.  The  dashed  lines  are  from  the  curve  [e~*  -  e'4x~]  * 

that  fits  the  shape  of  a  psy  chophy  sical  channel  determined  from  the  adaptation  studies  of  Blakemore  and  Campbell 
1 19691. 

Several  of  the  data  |a>ints  from  the  on-off  response  of  the  cortical  cell  (filled  circles)  were  shifted  slightlv  along  the 
horizontal  axis  to  allow  the  curves  to  coincide  with  the  zero  crossings  of  the  sine  function.  These  shifts  are  justified 
because  there  is  a  range  of  different  size  channels;  in  general,  they  represent  a  change  in  size,  and  the  main  concern  is 
with  the  relative  amplitudes  of  the  central  maximum  and  side  lobes. 

There  is  good  agreement  between  the  general  shape  of  these  curves.  The  sine  function  usually  has  lower  side  lobes  (less 
inhibition  I,  but  it  extends  farther  in  space  than  the  channel  and  cell  data.  The  greatest  amount  of  inhibition  is  seen 
in  the  psychophysical  channel.  It  should  be  pointed  out  that  prefiltering  the  spectrum  with  an  MTF  filter  and  then 
using  the  ideal  low-pass  filter  will  increase  the  inhibition  of  the  oorrespot  Jing  line-spread  function.  An  ideal  band-pass 
filter  will  further  increase  the  inhibition,  as  can  be  seen  by  comparing  the  line-spread  function  in  Figure  4,b,  I  with  4.b.3. 
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Are  the  two-dimensional  ideal  band-pass  filters  neurophysiologically  realizable?  The  corresponding  point-spread 
functions  of  ideal  circle-  and  square-shaped  spatial  filters  were  shown  in  Figure  6.  The  point-spread  function  of  the 
circle-shaped  filter  is  called  a  Bessel  function  of  the  first  kind,  order  zero  le.g.,  see  Hecht  and  Zajac,  19741.  It  is  similar 
to  the  isotropic  receptive  fields  found  in  the  retina  and  LGN. 

The  square-shaped  spatial  filter  has  vertical  and  horizontal  line-spread  functions  that  are  given  by  the  sine  function.  It 
can  be  viewed  as  a  generalized  channel  or  as  the  superposition  of  several  channels  having  the  same  or  different 
bandwidths  with  different  orientation. 

By  intuition,  one  might  expect  that  the  different  two-dimensional  shapes  of  filters  having  similar  bandwidths  would 
greatly  affect  the  shape  of  objects.  However,  Ginsburg  1 1971 , a, b)  has  shown  that  although  there  are  discriminable 
differences  between  objects  that  are  optically  low-pass  filtered  with  square,  circular,  and  rectangular  1 1 :0.6I  filters 
having  similar  bandwidths,  the  objects  can  still  be  easily  classified  as  the  same.  Similar  results  will  be  shown  later  using 
digital  filters  having  different  shapes  and  weighting  functions. 

A  diagram  of  an  ideal  low-pass  digital  transform  filter  that  passes  the  first  four  harmonics  is  shown  in  Figure  IK 
(Ginsburg,  1971, a  I.  This  filter  has  a  ±  1  octave  band-pass  centred  at  2  c/width  of  the  sampling  grid.  The  orientation 
selectivity  of  this  kind  of  digital  filter  is  given  by  45/n  degrees  where  n  is  the  number  of  the  n1^  harmonic.  From  Figure 
IK.  it  can  be  seen  that  the  orientation  selectivity  required  to  pass  the  first,  second,  third,  and  fourth  harmonics  is  45. 
22.5,  15,  and  1 1.25  degrees,  respectively. 

The  reader  is  reminded  that  the  bandwidth  and  orientation  selectivity  required  by  such  digital  filters  are  easily  found 
in  both  neurophysiological  and  psychophysical  data  from  cat,  monkey,  and  man,  as  was  discussed  in  Chapters  6  and  7. 

In  summary,  very  simple  band-pass  filters  in  the  Fourier  transform  domain  were  shown  to  provide  filtering  properties 
whose  counterparts  in  the  space  domain  exhibit  the  general  filtering  properties  of  single  channels  in  the  visual  system. 
The  next  chapters  show  how  these  filters  demonstrate  possible  visual  processing  of  spatial  information  of  simple  and 
complex  objects. 


PART  III 


VISUAL  INFORMATION  PROCESSING 
IN  TERMS  OF  SPATIAL  FILTERING 


“Percepts  without  concepts  are  blind;  concepts  without  percepts  are  empty." 

Immanuel  Kant 
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CHAPTER  9 


SPATIAL  FILTERING  AND  THE  GENERALIZATION 
OF  FORM :  SIMPLE  CONTINUOUS  AND  DISCRETE  OBJECTS 

9.1  Introduction 

The  preceding  sections  presented  the  theoretical  background  and  experimental  data  needed  to  create  certain  spatial 
filters.  These  spatial  filters  are  constrained  by  biological  data  and  are  used  to  determine  the  spatial  information  about 
objects  which  can  be  extracted  by  those  filters. 

This  dissertation  deals  with  only  a  few  of  the  ranges  of  objects  studied.  The  examples  discussed  have  been  selected  to 
bring  out  distinct  problems  for  space  domain  pattern  recognition  techniques  or  because  they  represent  a  particular  class 
of  perceptual  problem.  In  general,  these  objects  are  similar  to  those  used  in  previous  investigations  of  spatial  filtering 
using  optical  spatial  filtering  techniques  (Ginsburg,  1971, a, bl.  The  results  presented  here,  in  addition  to  replicating  the 
previous  results,  also  extend  them. 

Space  domain  pattern  recognition  means  the  recognition  of  patterns  by  the  explicit  extraction  and  manipulation  of 
pattern  features  such  as  lines,  angles,  or  edges  without  considering  how  the  visual  system  filters  the  various  spatial 
frequencies  that  are  present  in  those  patterns.  In  contrast,  the  transform  filtering  approach  extracts  information  from 
patterns  as  a  whole,  that  is,  extracts  pattern  information  in  a  way  that  interrelates  all  of  the  pattern  features.  That 
information  is  then  filtered  into  different  ranges  of  spatial  frequency  and  orientation  that  are  based  on  the  filtering 
processes  of  the  visual  system.  The  reader  is  encouraged  to  test  the  parsimony  of  the  filter  results  against  other  possible 
space  domain  techniques. 

The  usage  of  classification,  recognition,  and  identification  is  sometimes  blurred.  For  example,  the  classification  of 
objects  by  machines  is  commonly  termed  pattern  recognition.  The  following  general  definitions  of  detection, 
classification,  recognition,  and  identification  are  used.  Detection  is  the  perception  of  the  minimum  amount  of  spatial 
information  about  an  object  with  which  a  performance  measure  can  be  obtained;  e.g.,  an  object  is  seen  75  percent  of  the 
presentation  time.  Classification  is  the  perception  of  that  spatial  information  required  to  place  an  object  in  a  group, 
e.g.,  "The  object  is  a  face."  Recognition  is  the  perception  of  the  spatial  information  required  to  determine  whether  an 
object  has  been  seen  before,  e.g.,  "That  is  the  same  face  seen  yesterday."  Identification  is  the  perception  of  that 
information  requir'd  to  name  a  particular  object,  e.g.,  "That  is  Mary's  face." 

Some  questions  asked  in  this  section  are;  What  are  the  effects  of  the  MTF  filters  on  the  details  of  objects?  What  does 
the  low  frequency  attenuation  of  the  MTF  do  to  low-pass  filtered  images?  What  is  the  effect  of  removing  very  low 
spatial  frequencies  from  a  filtered  image,  i.e.,  using  a  band-pass  filter?  Finally,  what  is  the  minimum  number  of  spatial 
frequencies  needed  to  classify  an  object? 

In  these  next  sections  it  is  important  to  keep  in  mind  how  the  spatial  frequencies  relate  to  the  dimensions  of  the  objects. 
A  measure  is  needed  to  relate  the  spatial  frequencies  of  an  object  independent  of  its  viewing  distance.  The  fundamental 
frequency,  or  first  harmonic,  will  refer  to  one  cycle  across  some  given  size  dimension  of  the  object.  For  example,  the 
fundamental  frequency  of  a  letter  width  means  one  cycle  across  the  width  of  the  letter.  Referring  back  to  Figures  I  and 
2.  if  the  width  of  the  letter  is  the  same  as  the  width  of  the  pictures  that  contain  one  cycle,  then  the  sine  and  cosine 
functions  provide  spatial  information  about  one  harmonic  of  the  letter  in  the  vertical  orientation.  The  magnitude  of 
this  vertical  harmonic  could  be  placed  in  the  first  horizontal  square  in  the  two-dimensional  spectrum.  Rotating  this 
fundamental  harmonic  45  and  90  degrees  provides  the  magnitude  of  the  fundamental  harmonic  for  the  other  two 
orientations  Isee  Figure  181. 

For  the  following  filtered  objects,  only  the  filter  size  will  be  given,  not  the  number  of  harmonics  that  the  filter  passes. 
Since  the  particular  bandwidth  required  to  show  various  effects  is  not  critical  (Ginsburg,  1  ‘>7 1  ,a.b  and  Ginsburg,  ct  al., 
19701.  the  filter  size  will  be  a  simpler  measure  to  use.  However,  the  fundamental  frequency  will  be  given  for  the  width 
of  selected  objects  and  the  actual  spatial  frequencies  that  are  passed  by  the  filter  can  be  easily  computed.  The 


4.1 


fundamental  frequency  of  the  object,  f0l<  is  given  by  the  number  of  pels  of  the  sampling  grid  divided  by  the  number  of 
pels  of  some  dimension  of  the  object.  For  example,  theletterT  in  Figure  19,a  is  15pelslong.  The  sampling  grid  is  32 
pels.  Thus,  fQi  =  32/15  =  2. 13  cycles/sampling  grid  (c/gl,  f02  =  4.26  c/g,  and  f0:i  =  6.39  c/g. 

The  particular  harmonic  of  the  object,  fhn>  given  by  the  fundamental  spatial  frequency  in  the  magnitude  spectrum,  is 
the  size  of  object  divided  by  the  grid  size,  or  the  inverse  of  fQ  | ,  multiplied  by  the  number  of  the  harmonic.  For  example, 
the  first  harmonic  of  the  magnitude  spectra  of  the  letter  T  in  Figure  19, a  is  f hi  =  15/32  =  0.47  cycles  across  the  object 
(c/ol.  Thus.  fh2  =  0.94  c/ o,  fh3  =  1 .4  c/ o,  etc.  It  can  be  seen  from  these  calculations  that  a  5  by  5  low-pass  spatial 
filter  will  capture  more  than  the  second  but  not  more  than  the  third  harmonic  of  the  object.  Thus,  a  5  by  5  means  a 
low-pass  filter  that  captures  5  x  0.47  or  2.35  spatial  frequencies  of  the  object  in  the  u,  v  dimensions  of  the  transform 
plane. 

1  he  important  point  is  that  low  spatial  frequencies  will  always  refer  to  the  low  spatial  frequencies  of  the  object  relati.e 
to  the  fundamental  spatial  frequency  of  some  dimension  of  the  object.  Generally,  they  will  refer  to  the  bandwidth  of 
the  lowest  four  harmonics. 

9.2  Letter  T 

The  effects  of  the  MTF  filter  on  a  very  simple  two-dimensional  pattern,  a  letter  T,  are  shown  in  Figure  19.  (This  pattern 
also  represents  a  version  of  the  horizontal-vertical  line  illusion,  as  is  discussed  in  Section  12.3.3. 1  The  filtered 
magnitude  spectra  are  shown  on  the  left  side  and  the  filtered  images  on  the  right.  There  are  only  slight  differences 
between  the  brightness  distribution  of  the  T  from  the  MTF(L),  Figure  19, b,  and  MTF(Hl,  Figure  19, c,  spatial  filters. 
Both  filters  result  in  a  substantial  reduction  in  contrast  of  the  vertical  and  horizontal  lines  and  create  maximum 
intensities  at  their  intersections.  The  vertical  lines  are  slightly  brighter  than  the  horizontal  lines,  most  noticeably  as  the 
vertical  line  extends  away  from  the  intersection.  Note  that  these  results  (and  other  results  to  follow )  suggest  that  the 
differences  between  the  low  frequency  attenuation  characteristics  of  the  two  filters  have  only  a  small  effect  on  the 
overall  shape  of  the  object.  However,  the  MTF(H )  filter  provides  slightly  more  homogeneity  in  the  brightness 
distribution  of  the  lines  than  the  MTF(L)  filter. 

It  is  important  for  the  reader  to  note  that  the  brightness  distributions  shown  in  the  pictures  in  this  dissertation 
accurately  represent  the  results  of  the  digital  filtering  and  are  not  artifacts  of  the  filtering  characteristics  of  his  own 
visual  system  (assuming,  of  course,  that  the  reader  is  an  emmetropel.  The  fidelity  of  these  pictures  to  the  numbers  used 
to  create  them  is  shown  in  the  two-  and  three-dimensional  contour  plots  of  Figure  20.  Careful  study  of  these  contour 
plots  and  the  pictures  will  show  excellent  agreement  between  the  peaks  of  the  contours  and  the  grey  levels  in  the 
pictures.  The  similarity  between  the  shape  and  detail  of  these  filtered  pictures,  whether  amplitude  or  intensity  values 
are  used,  is  shown  by  comparing  Figure  19  with  Figure  20.  Thus,  such  nonlinearities  as  occur  between  an  amplitude 
and  an  intensity  representation  of  a  filtered  image  will  have  little  effect  on  the  overall  two-dimensional  shape  and  the 
amplitude  relationship  between  different  parts  of  the  filtered  image. 

The  MTF  filters  act  as  differentiators  at  the  low  spatial  frequencies.  This  results  in  a  reduction  in  contrast  between  the 
original  letter  T  and  the  MTF  filtered  Ts.  In  order  to  enable  the  details  of  the  filtered  images  to  be  seen  better,  the 
intensity  values  of  the  MTF  filtered  images  were  multiplied  by  a  logarithmic  (log)  function.  This  operation  does  not 
alter  the  general  structure  of  the  object;  it  just  increases  the  contrast  to  make  it  easier  to  visualize  details  of  the  object. 
I’he  log  transform  increases  the  contrast  of  the  low  contrast  details  of  the  images  by  multiplying  the  intensity  values  of 
the  images  by  a  logarithmic  function  scaled  between  the  extremes  of  the  intensity  values  0  (whitel  and  63  (black  I. 
Multiplication  of  the  values  of  the  intensity  by  the  log  function  results  in  increasing  the  lowest  intensities  to  a  greater 
extent  than  the  highest  intensities.  The  overall  effect  is  to  compress  the  larger  range  of  intensities  of  the  images  into  a 
more  narrow  range  that  becomes  more  visible  with  the  limited  range  of  intensity  than  was  captured  on  the  Polaroid  film 
that  was  used.  The  log  transform  increases  the  lower  intensities  while  preserving  the  overall  smoothing  effects  of  the 
filter  on  the  object.  This  can  be  verified  by  comparing  the  log  transformed.  MTF  filtered  letter  T  in  Figure  19,d  with 
the  10-level  contour  plot  of  Figure  20, b. 

The  values  of  the  MTF  filtered  spectra  have  also  been  log  transformed  in  order  to  increase  the  visibility  of  the  low- 
contrast  details  of  the  spectra.  Compare  the  spectra  in  Figure  19.r  and  d. 


What  is  the  minimum  number  of  spatial  frequencies  required  to  identify  this  letter  TV  Also,  what  are  the  effects  of  the 
MTFl  Ll  and  MTFl  H I  filters  on  the  low  spatial  frequencies?  Both  of  these  questions  are  answered  by  the  next 
demonstrations  in  which  the  letter  T  is  synthesized  from  1  to  6  c/o  using  both  filters.  The  results  are  shown  in  Figures 
21  and  22  in  which  a  T-shaped  object  can  be  recognized  from  about  1.5  to  almost  3  c/o  (Figures  21  and  22,c,d,el. 

There  is  little  difference  between  the  shapes  of  these  filtered  images  in  these  bands  of  spatial  frequency.  This  result 
implies  that  the  particular  bandwidth  required  to  provide  the  basic  form  of  an  object  is  not  critical  over  a  range  of  at 
least  +  I  c/o. 

A  final  demonstration  shows  that  an  anisotropic  spatial  filter  and  the  removal  of  very  low  spatial  frequencies  of  the 
letter  T  still  provide  sufficient  information  for  the  identification  of  the  object.  The  results  of  filtering  the  original  'I'  with 
a  rectangular  filter,  flu.  vl  =  6,  4,  with  and  without  the  low  spatial  frequencies  removed  tfhl  =  0.47  c/o)  are  shown  in 
Figure  lO.f.g.  The  spectra  and  filtered  images  have  not  been  log  transformed  because  the  MTF  filters  were  not  used. 
Note  that  rectangular  filtered  letter  T  in  Figure  10. f  is  comparable  to  the  MTF  filtered  letter  T  in  Figure  10, e  even 
though  quite  different  spatial  filters  have  been  used. 

This  invariance  of  form  with  changes  in  the  filter  characteristics  using  filters  that  can  be  related  to  mechanisms  in  the 
visual  system  will  be  shown  several  times  with  many  different  objects. 

The  major  differences  between  the  objects  filtered  by  the  two  MTF  filters  are  the  magnitudes  of  the  ripples  distant  from 
the  object  that  can  be  seen  in  several  of  the  contour  plots.  The  MTF ( L I  filter  produces  ripples  of  greater  magnitude 
than  does  the  MTFIH I  filter.  This  is  due  to  increased  inhibition,  i.e.,  the  greater  low  frequency  attenuation  of  the 
MTFl  LI  filter.  In  general,  these  ripples  are  of  relatively  small  value.  For  example,  the  ripples  seen  in  the  photograph  of 
the  low-pass  filtered  T  (Figure  l*>,e I  are  not  seen  in  the  contour  plot  of  the  same  object  in  Figure  22, f.  This  means  that 
those  ripples  are  less  than  10  percent  of  the  maximum  value. 

Now  that  one  object  has  been  filtered  in  detail,  only  the  general  results  of  similar  filtering  of  other  objects  will  be  shown. 
In  general,  only  the  MTFIH  1  spatial  filter  will  be  used  to  represent  the  filtering  effects  of  the  two-dimensional  contrast 
sensitivity  function  on  objects.  The  small  differences  between  the  MTFlLl  and  MTFlH  I  filters  are  more  important  for 
contrast  illusions  than  general  form  information,  as  is  discussed  in  Chapter  12.  Numerical  printouts  and  contour  plots 
will  be  used  only  to  make  specific  points  about  certain  objects  that  are  not  immediately  apparent  from  the  pictures. 

The  results  of  filtering  more  complex  objects  are  shown  in  Figure  23.  The  original  objects  are  shown  in  Column  1 .  The 
MTFIH  I  filtered  spectra  are  shown  in  Column  2.  Column  3  shows  the  MTFlH  I  filtered  images.  Columns  4,  .">.  and  (i 
show  the  low-pass  filtered  images.  Column  4  shows  filtered  images  containing  only  four  cycles  prefiltered  with  the 
MTFl  H I  followed  by  an  ideal  low-pass  filter.  An  ideal  low-pass  filter  was  used  to  capture  four  cycles  of  the  filtered 
images  shown  in  Column  5.  The  filtered  images  in  Column  (>  contain  only  three  harmonics  after  removal  of  the  lowest 
harmonic  (about  the  0.5  cycle  of  the  object)  and  those  harmonics  greater  than  four. 

0.3  Letter  K 

The  MTFlH  I  filter  does  little  to  the  letter  E  (Figure  23, a,  1  (,  except  to  smooth  the  corners  and  lines  I  Figure  23,a,3l. 
Except  for  contrast,  there  is  little  difference  between  the  form  of  the  letter  E  created  from  four  harmonics  with  I  Figure 
23. a, 41  and  without  (Figure  23, a,  5)  the  MTFl  HI  filter.  Intuition  suggests  that  the  lowest  harmonic  of  the  letter  should 
provide  just  contrast,  not  form,  information.  That  intuition  is  confirmed  in  Figure  23,a,(>  where  the  removal  of  the 
lowest  harmonic  has  reduced  contrast  but  has  had  little  effect  on  the  form  as  compared  to  the  low-pass  filtered  Es  of 
Figure  23. a. 4,. 5.  Thus,  only  three  harmonics  are  required  for  the  identification  of  this  letter  E. 

‘1.4  Discrete  Letter  (i 

This  next  ex|>eriment  investigates  how  objects  that  are  made  of  discrete  elements  can  be  perceived  as  a  whole  unit  of 
form  if  man  or  machine  can  isolate  the  low  spatial  frequencies.  The  discrete  squares  of  the  letter  ( i  in  Figure  23. b.  I 
jierceptually  close  to  provide  the  (J  form  information,  ('.an  the  filter  concept  offer  any  clues  as  to  how  a  global  form 
could  emerge  from  such  discrete  pattern  elements?  First,  the  effects  of  the  MTF  spatial  filter  on  the  discrete  pattern 
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are  noted.  Figure  23. b. 3  shows  that  the  MTF  filter  has  only  smoothed  the  squares.  They  can  still  be  resolved  even  after 
the  log  transform.  Thus,  a  global  G  has  not  emerged  by  filtering  the  original  pattern  according  to  the  contrast 
sensitivity  data.  In  other  words,  the  G  form  has  not  been  generalized  from  its  discrete  elements  by  the  single  overall 
spatial  filtering  characteristics  of  the  visual  system  as  processed  by  the  MTF  filter  that  has  been  used.  However,  when 
the  higher  spatial  frequencies  of  the  discrete  G  pattern  are  sequentially  removed,  the  square  dots  are  not  resolved  and 
the  generalized  G  form  is  found  in  three  ranges  of  low  frequency  from  1  to  ,1.  I  to  4,  I  to  3  c/o  (Figure  24,c,d.e(.  The 
best  subjective  G  form  is  present  in  a  frequency  band  of  1  to  4  c/o  (Figures  23,b.4,3,  and  24. d  I.  Similar  results  are 
obtained  w  ith  the  ideal  low-pass  filter.  Kemoving  the  fundamental  frequency  demonstrates  once  again  a  reduction  in 
contrast  in  the  filtered  pattern.  The  basic  form,  however,  remains  with  only  2  to  4  c/o  shown  in  Figure  23. b. 6. 

Can  the  generalized  G  form  be  found  in  a  range  of  high  spatial  frequencies?  The  answer  to  this  question  is  "no 7, 
noting  the  results  of  sequentially  filtering  out  the  lower  spatial  frequencies  as  shown  in  Figure  2.7.  Thus,  excluding  the 
lower  spatial  frequencies  results  in  either  the  dots  or  nondescript  forms.  Only  a  band-pass  region  of  low  spatial 
frequencies  contains  the  generalized  form. 

‘1.4. 1  Similar  Forms  from  Four  Discriminahly  Different  Letter  Gs 

The  previous  results  suggest  that  the  discriminable  differences  between  objects  are  removed  by  low-pass  filtering  and 
that  the  particular  elements  that  make  up  the  object  are,  in  general,  irrelevant  for  the  classification  of  the  object.  This 
suggestion  is  shown  to  have  considerable  merit  in  the  following  demonstration.  Four  discriminahly  different  letter  (is. 
Full  G,  Dot  G.  Serif  G,  and  Slash  G.  are  shown  to  have  similar  forms  when  created  from  their  low  spatial  frequencies  in 
Figure  26. 

6.3  Dot  Letter  K  in  Noise 

Similar  results  were  obtained  with  a  figure-ground  problem  of  a  dot  R  buried  in  random  dot  noise  ( Figure  2.'i,c.  1 1. 

Clearly  ,  any  system  that  analyzes  this  object  in  terms  of  local  features  will  have  difficulty  in  discriminating  the  dots  of 
the  R  from  the  noise.  However,  separating  the  R  form  from  the  noise  by  spatial  filtering  is  simple  because  their  main 
spectral  energy  occurs  in  different  spatial  frequency  bands  ( Figure  23, c. 43. 61. 

Of  course,  using  the  spatial  filter  to  extract  embedded  figures  would  be  difficult  if  the  size  of  the  noise  were  to  approach 
the  size  of  the  elements  making  up  the  object.  But  it  should  be  noted  that  the  human  observer  would  also  have  increased 
difficulty. 

6.6  Gestalt  Principles  of  Perceptual  Organization 

The  preceding  experiments  have  demonstrated  that  a  low  spatial  frequency  band  that  contains  only  three  harmonics  is 
sufficient  to  provide  generalized  form  information  of  objects  that  have  names,  i.e.,  that  can  be  classified.  The  next 
experiment  investigates  the  effects  of  spatial  filtering  on  more  general  dot/line  patterns.  These  patterns  illustrate 
fundamental  laws  of  perceptual  organization  enunciated  by  early  Gestalt  psychologists:  whole  (Figure  23,d.l  I. 
proximity  (Figure  23, e,  1 1,  similarity  (Figure  2.3, f.  1 1  and  proximity/similarity  (Figure  23. g. 1 1.  Pattern  elements 
perceptually  close  and  group  to  form  units  of  information  different  from  the  sum  of  their  parts,  e.g..  Koffka  (16331. 

As  before,  the  MTF  spatial  filter  just  smooths  the  pattern  elements  and  the  low-pass  spatial  filter  isolates  the  basic 
forms.  Thus,  the  perceived  groupings— uniform  field  for  the  whole,  three  uniform  groups  for  the  proximity .  two  similar 
groups  separated  by  a  central  dissimilar  group  for  the  similarity,  and  three  uniform  groups  for  the  proximity  /similarity  — 
can  be  found  in  a  low  spatial  frequency  band,  f  =  1 . 4  c/o  (Figure  23.d-g.4-6l.  Note  that  the  removal  of  the  lowest 
harmonic  of  the  dot  pattern  has  the  additional  benefit  of  providing  generalized  grouping  information.  It  has  also 
reduced  the  contrast  difference  between  dissimilar  pattern  features  in  Figure  23.d-g,6.  There  are  many  other  examples 
of  closure.  For  example.  Figure  27.  a  shows  a  letter  H  made  up  of  line  segments.  The  low-spatial  frequencies  of  this 
object  provide  the  completed  form  as  shown  in  Figure  27. c. 

6.7  Summary 

The  previous  sections  have  shown  that,  in  general,  the  basic  form  of  many  different  objects  can  be  seen  from  only  three 


harmonics  of  the  original  object.  Can  such  simple  but  powerful  filtering  also  work  on  more  complex  objects  such  as 
portraits?  This  is  investigated  in  the  next  chapter. 
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CHAPTER  10 


SPATIAL  FILTERING  AND  THE  GENERALIZATION 
OF  FORM:  COMPLEX  OBJECTS  (VARIATIONS  OF  A  PORTRAIT) 

10.1  Introduction 

The  previous  experiments  have  shown  that  spatial  filters  based  on  biological  data  can  provide  details  and  general  forms 
of  simple  objects  that  agree  well  with  the  way  the  visual  system  seems  to  filter  and  extract  information  about  objects.  In 
this  section  a  more  complex  object  is  investigated — a  portrait. 

An  original  portrait  (Figure  28. a.  1 1  is  used  as  a  control  in  the  following  demonstrations.  Four  variations  of  this  original 
portrait  have  been  made:  coarse  quantization  (Figure  28.b,l  I,  added  random  noise  (Figures  28.C.1 ;  31,a.l  I.  inclusive 
"OR"  random  noise  (Figure  3 Lb,  1 1,  and  line  configuration  (Figure  31.C.1 1.  Each  portrait  is  discriminably  different 
from  the  others  and  yet  is  able  to  convey  similar  general  face  information.  As  in  the  previous  demonstrations,  the 
fidelity  of  the  MTF  will  be  noted  in  determining  the  resolution  of  the  portrait  features  that  are  seen.  The  low  spatial 
frequencies,  as  predicted  from  the  previous  filtering  results,  will  be  examined  for  similar  portrait  form  information. 
Answers  to  several  questions  will  be  sought  as  the  effects  of  the  MTF  and  low-pass  filters  are  observed:  Why  do  the 
portrait  features  look  different  at  different  viewing  distances?  How  do  these  portrait  configurations  convey  similar  face 
information?  What  is  the  minimum  amount  of  information  needed  to  recognize  and  identify  a  face? 

Harmon  (  P1T3I  provided  an  interesting  investigation  concerned  with  the  recognition  of  faces  using  both  blurring  and 
coarse  quantization.  His  basic  findings  were  that  60  percent  correct  face  recognition  can  be  obtained  even  under  severe 
blur  and  that  a  16  by  16  matrix  of  quantized  squares  was  close  to  the  minimum  resolution  required  for  identification. 
Unfortunately,  it  is  difficult  to  translate  the  space  averaging  blur  that  he  used  into  spatial  frequency  terms.  Clearly,  it 
is  not  band-pass  filtering.  Furthermore,  the  16  by  16  matrix  that  provided  the  minimum  resolution  for  identification 
appears  to  be  related  to  the  sampling  grid  and  not  relevant  portrait  dimensions.  A  16  by  16  sampling  matrix  can  provide 
a  maximum  of  eight  harmonics.  However,  previous  filtering  results  predict  that  only  about  four  harmonics  should  be 
necessary  for  face  identification.  An  examination  of  the  coarsely  quantized  portraits  that  were  shown  in  Harmon's 
article  reveals  an  average  of  1(1  blocks  across  the  face.  Thus,  the  sampling  that  was  used  provided  about  five  harmonics 
|>er  face  width,  a  value  more  in  agreement  with  predictions. 

In  another  study.  Harmon  and  Julesz  ( 16731  investigated  the  masking  effect  of  visual  noise  on  the  perception  of  a 
portrait  by  coarsely  quantizing  a  portrait  of  Abraham  Lincoln  into  large  blocks.  Their  general  conclusion  was  that 
noise  due  to  the  high  frequencies  of  the  blocks  masked  the  low  frequencies  that  carry  the  portrait  information.  They 
termed  this  effect  "critical-band  masking."  In  one  demonstration,  they  removed  a  two-octave  band  of  frequencies  just 
above  the  block  frequency.  The  result  was  a  low  frequency  portrait  that  also  contained  the  edges  of  the  blocks.  The 
face  was  more  easily  recognized  in  comparison  to  a  face  in  which  the  removal  of  some  high  frequencies  eliminated  the 
sharp  edges.  However,  the  recognition  of  that  filtered  face  was  not  so  good  as  would  be  expected  taking  into 
consideration  that  the  edge  frequencies  are  two  octaves  higher  than  the  face  frequency  and  that  psychophysical  studies 
(see  Chapter  61  have  shown  spatial  frequency  interaction  within  about  a  two-octave  bandwidth.  In  a  further 
demonstration,  they  added  random  noise  that  was  spectrally  adjacent  to  but  two  octaves  higher  than  that  of  the  blurred 
portraits  of  Lincoln.  Again  the  perceived  differences  between  the  two  portrait  configurations  were  not  so  great  as  one 
would  expect  from  critical-band  masking.  These  demonstrations,  especially  from  the  latter  results,  imply  that 
perceptual  mechanisms,  in  addition  to  frequency  specific  channels,  may  cause  or  contribute  to  our  difficulty  in 
recognizing  coarsely  quantized  portraits. 

10.2  Filtering  the  Original  Portrait 

An  original  continuous  tone  portrait  was  digitized  to  103  by  112  pels  containing  64  grey  levels  (>er  element  and 
processed  on  a  1 28  by  1 28  pel  grid.  The  width  of  the  face  was  about  64  pels.  Appropriate  a|)erture  ta|>ering  functions 
to  reduce  leakage  (see  Section  2.41  were  not  used  in  this  study  because  portraits  normally  do  have  borders  around  them 
and  it  is  interesting  as  an  aside  to  examine  leakage  effects  from  such  abrupt  pattern  edges. 
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The  amplitude  values  of  the  original,  variations  of  the  original,  and  the  filtered  portraits  were  normalized  to  the  same 
maximum  and  minimum  values  except  where  noted.  The  portraits  were  Fourier  transformed  by  a  digital  Fast  Fourier 
Transform  technique  (see  Section  3.1 ).  The  resulting  spectra  were  filtered  by  attenuating  according  to  a 
two-dimensional  contrast  sensitivity  function,  MTF(H  (,  and/or  ideal  low-pass  filtered.  The  filtered  portraits  were  then 
reconstructed  by  performing  an  inverse  Fourier  transform. 

The  fundamental  frequency  is  attenuated  by  the  contrast  sensitivity  data  obtained  at  0.5  c/d.  Thus  the  MTF  filtered 
portraits  should  be  viewed  at  about  2  degrees  of  visual  angle  (about  1.2  meters)  to  determine  the  fidelity  of  the  MTF 
filtered  picture  to  one’s  own  visual  filtering  characteristics. 

The  reconstructed  portraits  were  enlarged  to  420  by  448  pels  by  creating  a  4  by  4  block  of  uniform  intensity  for  each  of 
the  105  by  1 12  pels.  Pictures  of  both  one-  and  two-dimensional  spectra  were  obtained.  A  log  transorm  of  these  spectra 
was  performed  in  order  to  increase  the  contrast  of  the  low  energy  spectral  components  on  the  film  (see  Section  0.21. 
The  spectra  represent  +  64  spatial  frequencies  in  two  dimensions.  The  fundamental  frequency  of  the  face  is 
f  1  =  2  c/picture  width  (c/pw). 

The  one-  and  two-dimensional  magnitude  spectra  of  the  portraits  in  Figure  28  are  shown  in  Figure  29.  The 
one-dimensional  magnitude  spectra  represent  the  spectra  along  the  vertical  axis  of  the  adjacent  two-dimensional 
spectra.  The  periodic  structure  in  the  transforms  is  due  to  truncation  from  a  23  by  16  pel  border  around  the  portrait 
that  occurred  when  the  original  portrait  was  digitized  to  105  by  1 12  pels  for  a  128  by  128  grid.  The  effect  of  this  artifact 
is  to  multiply  the  spectrum  of  the  portrait  by  the  spectrum  of  a  square  pulse  (e.g.,  see  Figure  3).  This  resulting  periodic 
structure  hinders  the  visual  analysis  of  the  spectrum  but  has  little  effect  on  the  filtered  image,  as  shall  be  shown.  Note 
the  elliptical  spectral  form  in  all  the  two-dimensional  spectra  around  the  low  spatial  frequencies  that  is  due  to  the  basic 
shape  of  the  face.  Also  note  the  relatively  lower  amplitude  of  the  high  frequency  ray-like  spectral  forms  (Figure  20, a  I 
due  to  the  details  of  the  original  portrait. 

The  filtered  portrait  in  Figure  28, a. 2  shows  general  smoothing  of  the  portrait  detail  due  to  the  high  frequency 
attenuation.  Note  that  there  is  no  sharpening  of  the  edges  from  the  MTF  filter.  When  these  portraits  are  viewed  at  the 
appropriate  distance  of  2  degrees  of  visual  angle,  the  resolution  of  the  details  of  the  original  and  MTF  filtered  portraits 
is  in  good  agreement.  Note,  for  example,  the  hair  patch  and  the  eyebrows.  This  result  suggests  that  the  shape  of  the 
MTF,  even  though  primarily  derived  from  threshold  contrast  sensitivity  data,  does  provide  a  reasonable  first-order 
approximation  of  our  visual  filtering  characteristics  over  about  2  degrees  of  visual  angle  for  suprathreshold  vision. 

The  portrait  information  that  is  in  the  low  frequencies  is  examined  next.  The  previous  filtering  results  suggest  that 
about  four  harmonics  should  be  required  for  the  general  face  information.  The  face  covers  about  half  the  width  of  the 
portrait.  Therefore,  8  c/ pw,  4  c/facelf),  should  provide  the  general  form  of  the  face.  The  results  of  such  low-pass 
filtering  are  shown  in  Figure  28, a, 3  where  a  very  smoothed  portrait  is  seen,  as  would  be  expected  from  low-pass 
filtering. 

The  same  question  that  was  asked  in  the  previous  chapter  on  the  simple  forms  is  asked:  Do  the  low  frequency 
attenuation  characteristics  of  the  MTF  affect  this  important  range  of  frequencies?  This  question  is  answered  by 
low-pass  filtering  the  portrait  without  prefiltering  with  the  MTF.  The  result  is  shown  in  Figure  28, a. 4  where,  except 
for  an  increase  in  contrast,  the  forms  are  almost  identical.  Therefore,  it  is  concluded  that  the  low  frequency  attenuation 
characteristics  from  the  MTF — moderate  differentiation  up  to  6  c/d  in  this  case — do  not  impair  the  basic  form  of  even 
complex  portraits. 

10.3  Filtering  Coarsely  (Quantized  Portraits 

Coarsely  quantized  spatial  information  greatly  reduces  the  amount  of  information  that  can  be  extracted  from  an  object. 
It  is  a  crude  form  of  low-pass  filtering.  There  are  certain  v  isual  displays  that  present  coarsely  quantized  objects.  It  is 
important  to  determine  how  much  usable  information  can  be  extracted  from  such  displays  iGinsburg.  10731. 

Harmon  1 10731  and  Harmon  and  Julesz  1 1073)  have  previously  demonstrated  that  a  coarsely  quantized  portrait  of 
Abraham  Lincoln  contained  sufficient  feature  resolution  for  recognition.  The  coarse  quantization  of  a  portrait  is,  by 
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itself,  a  low-pass  spatial  filter  operation  that  replaces  the  original  spectral  information  with  spectral  energy  at  the 
fundamental  sampling  frequency  of  the  coarsely  quantized  block  and  its  harmonics.  Harmon  noted  that  recognition  of 
the  coarsely  quantized  portrait  is  improved  by  squinting,  defocusing.  or  by  viewing  at  a  distance.  These  results  suggest 
that  the  noise  energy  introduced  at  the  higher  spatial  frequencies  by  the  coarsely  quantized  blocks  masks  the  perception 
of  the  low  spatial  frequency  form  information  (Harmon  and  Julesz,  19731.  Reducing  that  noise  by  means  of  high 
frequency  attenuation  allows  the  low  spatial  frequency  form  information  to  be  more  readily  perceived. 

This  study  will  further  investigate  the  similarity  of  form  information  contained  in  the  original  and  coarsely  quantized 
portraits  by  means  of  low-pass  spatial  filtering.  Also,  as  in  the  original  portrait,  the  effects  of  the  MTF  attenuation 
characteristics  on  the  coarsely  quantized  portrait  will  be  shown. 

The  original  420  by  448  pel  portrait  was  coarsely  quantized  into  15  by  16  square  blocks  of  uniform  amplitude  values; 
each  block  averaged  over  28  by  28  pels.  The  result  is  shown  in  Figure  28,b,l  where  the  inhibited  perception  of  the 
portrait  is  observed. 

A  certain  amount  of  care  has  to  be  used  to  coarsely  quantize  a  portrait  to  ensure  that  the  original  features,  e.g.,  eyes, 
nose,  and  mouth,  are  sufficiently  sampled.  As  Harmon  ( 19731  noted,  overall  recognition  increased  from  about  10 
percent  to  95  percent  with  optimally  placed  sampling  blocks.  This  result  can  be  understood  in  terms  of  sampling 
theory.  There  are  15  and  16  blocks  in  the  horizontal  and  vertical  directions,  respectively.  The  sampling  theorem  states 
that  a  wave  function  must  be  sampled  by  at  least  twice  the  highest  frequency  that  is  to  be  unambiguously  recovered 
(reconstructed  (from  that  function.  Undersampling  the  original  portrait  by  the  coarse  blocks  produces  aliasing,  i.e.,  false 
frequency  energy  introduced  by  the  sampling  function  around  the  sampling  frequency.  Thus,  at  most,  7.5  and  8  c/ pw 
spatial  frequency  information  in  the  horizontal  and  vertical  directions  from  the  original  portrait  can  be  recovered, 
and  spurious  energy  that  does  not  contain  portrait  information  will  be  introduced  above  the  sampling  frequency.  This 
implies  that  the  placement  of  the  sampling  blocks  is  critical  in  order  to  preserve  accurately  the  low  spatial  frequency 
information  below  8  c/pw.  and  one  has  to  consider  the  particular  features  that  are  sampled  by  each  block.  Thus,  the 
sampling  blocks  should  be  placed  to  sample  the  relevant  low  spatial  frequency  information  of  the  original  pattern  —  in 
the  present  case,  the  eyes.  nose,  and  mouth,  etc.  —  in  order  to  maintain  the  original  low  frequency  spatial  information 
previously  demonstrated  to  carry  the  basic  form  information. 

The  one-  and  two-dimensional  log  transforms  are  shown  in  Figure  29, a, b,  1 ,2.  The  coarse  quantization  process  removes 
all  but  the  low  spatial  frequency  elliptical  structure  to  about  8  c/pw  and  replaces  the  original  high  frequency  oblique 
ray  structure  w  ith  periodic  rectangular  components  due  to  the  block  harmonics.  Similarity  exists  between  the  line 
spectra  of  the  original  and  coarsely  quantized  portraits  only  up  to  about  10  c/pw. 

The  effect  of  the  MTF  filter  on  this  portrait  is  shown  in  Figure  28,b,2.  The  MTF  spatial  filter  reduces  the  contrast, 
as  expected  from  the  low-spatial  frequency  attenuation  characteristics  of  the  filter,  and  only  local  smoothing  occurs. 
Observation  of  the  original  and  MTF  spatial  filtered  portraits  at  about  2  degrees  of  visual  angle  reveals  similar  feature 
resolution  even  though  substantial  contrast  reduction  is  evident.  Again  the  high  frequency  attenuation  of  the  MTF 
spatial  filter  appears  to  provide  a  good  first  approximation  for  resolution  of  features  under  suprathreshold  viewing 
conditions. 

MTF  spatial  filtering  followed  by  low-pass  spatial  filtering,  f  —  4  c/f.  results  in  the  coarsely  quantized  portrait  shown 
in  Figure  28. b. 3.  Figure  28,b,4  shows  the  results  of  low-pass  filtering  on  its  own.  appearing  very  similar  to  Figure 
28, b, 3  except  for  contrast.  Thus,  the  low  spatial  frequency  attenuation  characteristics  of  the  MTF — differentiation  up 
to  6  c/pw— have  little  affect  upon  the  low  spatial  frequency  form  information  of  this  portrait.  However,  a  comparison 
between  MTF  low-pass  spatial  filtered  original  and  coarsely  quantized  portraits  demonstrates  a  marked  decrease  in 
contrast  of  the  eyes.  nose,  and  mouth,  as  well  as  left  cheek  and  shoulder,  in  the  coarsely  quantized  portrait.  But.  again, 
these  results  should  be  tempered  with  the  realization  of  the  importance  of  the  block  placement  in  sampling  the  original 
pattern  features. 

10.3.1  Why  Does  ( hoarse  (Quantization  Reduce  Form  Perception? 

Although  Harmon  and  Julesz  1 19731  suggested  that  portraits  were  masked  by  the  noise  introduced  by  coarse 
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quantization,  there  would  seem  to  be  other  factors  that  reduce  the  perception  of  coarsely  quantized  portraits. 

First,  the  degree  of  quantization  in  both  this  study  and  that  of  Harmon  and  Julesz  11973)  occurs  at  the  sampling  limit 
for  the  basic  features  such  as  the  eyes,  nose,  mouth,  and  overall  shape.  In  addition  to  being  almost  impossible  to  sample 
such  general  features  to  the  same  degree  because  of  the  uniform  sampling  size  that  was  used,  such  coarse  sampling 
reduces  the  amount  of  spatial  information  about  the  portrait  to  almost  zero  and  introduces  aliasing.  The  lack  of  details 
(the  higher  spatial  frequencies  of  the  eyes,  etc.  I  and  the  result  of  aliasing  (spurious  patterns  resulting  from  spurious 
spectral  energy  I  can  be  seen  quite  clearly  by  comparing  the  original  and  coarsely  quantized  filtered  portraits  in  Figure 
30.  These  original  and  coarsely  quantized  portraits  have  been  low-pass  filtered  from  f  =  24,  16,  and  12  c/pw. 

A  second  factor  arises  if  it  is  accepted  that  mechanisms  in  the  visual  system  perform  piece-wise  Fourier-like  filtering 
over  finite  areas  of  an  object.  An  object  is  bounded  by  its  edges.  The  higher  the  contrast  of  the  edges,  the  more  likely 
it  is  that  the  object  has  distinguishable  boundaries.  In  such  a  scheme  each  block  and/or  multiples  of  blocks  with  similar 
amplitudes  that  make  up  the  coarsely  quantized  picture  would  be  considered  to  share  the  same  boundary  and  could  be 
analyzed  separately.  That  would  create  difficulty  in  determining  the  lower  harmonics  for  the  overall  form  of  the 
portrait.  This  point  is  discussed  further  in  Sections  13.3.1  and  19.9.3.1. 

A  third  factor  is  the  creation  of  Mach  bands  by  coarse  quantization  (Ginsburg,  1973).  Note  that  many  of  the  blocks  of 
low  contrast  that  are  adjacent  to  blocks  of  similar  contrast  in  Figure  28b,  1  do  not  appear  to  have  uniform  brightness 
distributions.  The  narrow  region  of  lighter  and  darker  brightness  distribution  along  the  edges  of  those  blocks  appears 
similar  to  the  brightness  distributions  of  Mach  bands.  Thus,  the  amplitude  of  the  original  blocks  will  not  be  completely 
uniform  and  the  average  values  will  change,  depending  upon  the  magnitude  of  the  Mach  bands.  This  will  further 
reduce  the  already  minimal  fidelity  of  spatial  information  from  sampling  considerations  about  the  object  that  could  be 
extracted  by  other  visual  processes. 

All  of  these  effects  need  to  be  investigated  further.  It  seems  clear,  however,  that  critical  band  masking  is  not  the  only 
factor  to  be  considered  in  such  problems. 

10.4  Filtering  a  Portrait  with  Random  Noise  Added 

This  next  demonstration  investigates  the  appearance  of  the  woman's  portrait  to  which  a  two-dimensional  random  noise 
pattern  with  a  uniform  amplitude  distribution  was  added.  The  amplitudes  of  the  pels  were  normalized  so  the  maximum 
amplitude  of  the  picture  did  not  exceed  the  limits  of  the  computer  range. 

The  result  of  adding  random  noise  to  the  original  portrait  is  shown  in  Figure  28,c,l  where  the  main  effect  of  the  noise 
is  to  greatly  reduce  the  detail,  particularly  in  the  eyes,  nose,  and  mouth. 

The  one-  and  two-dimensional  spectra  (Figure  29, c,  1,21  contain  very  little  structure  except  at  the  low  spatial 
frequencies,  below  8  c/pw,  where  the  two-dimensional  elliptical  structure  from  the  basic  face  form  persists.  The  line 
spectrum  reveals  a  profile  more  similar  to  the  original  line  spectrum  than  that  of  the  coarsely  quantized  portrait.  The 
spectra  again  suggest  that  the  basic  portrait  form  is  relatively  intact  below  f  =  4  c/f. 

The  effect  of  the  MTF(H)  spatial  filter  reveals  a  smoother,  more  cosmetically  appealing  portrait  (Figure  28. c,2)  with 
quite  similar  feature  resolution  of  the  eyes,  nose,  and  mouth,  although  with  increased  contrast. 

Low-pass  spatial  filtering  the  filtered  noise  portrait  with  and  without  MTF  prefiltering  to  f  =  4  c/f  (Figure  28,c„3,4l 
once  again  reveals  similar  form  information  virtually  unaffected  by  the  MTF  attenuation  characteristics. 

10.4. 1  Artifacts  from  a  Portrait  with  Random  Noise  Added 

It  is  important  to  realize  that  random  noise  can  be  correlated  with  pictures  in  specific  ways.  In  other  words,  the  same 
size  random  noise  with  a  constant  amplitude  distribution  that  results  in  a  uniform  amplitude  spectrum,  created  with 
different  random  values,  will  affect  the  original  picture  differently.  This  is  demonstrated  in  Figure  31  .a,  1  where 
another  random  noise  pattern  with  the  same  parameters  as  before  is  added  to  the  original  |*irtrait.  Note  especially  the 
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eyes,  nose,  mouth,  and  left  cheek  differences  between  this  portrait  and  that  of  Figure  28, c.  1 .  The  different  spectrum 
resulting  from  this  second  added  random  noise  portrait  is  aparent  upon  observation.  The  low-pass  spatial  filtered 
portrait  for  f  =  4  c/f.  shown  in  Figure  3 1  ,a, 3.  demonstrates  similar  form  information;  however,  note  the  difference 
between  the  resolution  of  the  hair  and  the  displaced  mouth  in  comparison  to  Figure  28,c,3.  These  results  suggest 
caution  in  ex|>eriments  based  upon  picture  correlated  random  noise,  because  even  those  features  that  contain  only  low 
spatial  frequencies  can  be  affected  differently  by  different  random  noise  obtained  from  a  similar  noise  generating 
process. 

10.3  Filtering  a  Portrait  with  Inclusive  OR  Random  [Noise 

The  portrait  with  added  random  noise  has  a  uniform  amplitude  distribution  that  will  increase  the  average  intensity  of 
the  portrait  if  the  average  noise  exceeds  that  of  the  original  portrait.  That  result  may  account  for  some  of  the  decreased 
contrast  of  the  pattern  information  after  spatial  filtering.  In  addition,  it  has  been  demonstrated  that  adding  different 
patterns  of  random  noise  with  a  uniform  amplitude  distribution  constructed  from  elements  of  the  same  size  affects  the 
basic  |«»rtrait  form.  'Phis  next  study  was  undertaken  to  reduce  the  average  pattern  intensity  as  well  as  to  combine  the 
same  random  noise  pattern  in  a  nonadditive  fashion.  The  desired  result  was  achieved  by  inclusive  OR-ing  the  original 
(xirtrait  with  the  random  noise,  as  shown  in  Figure  31,b.  1.  (The  inclusive  OR  is  a  logical  binary  bit  operation  that 
places  a  1  if  either  or  both  the  random  noise  value  and  the  portrait  value  have  a  1  in  the  same  picture  value  bit  position  I. 
Note  the  decreased  perception  ol  the  nose  and  mouth  in  comparison  to  the  added  noise  portraits. 

The  two-dimensional  log  magnitude  spectrum  (Figure  3l.b,2l  reveals  an  elliptical  low  spatial  frequency  structure 
similar  to  that  of  the  other  portrait  configurations.  In  addition,  spectral  energy  is  less  random,  as  well  as  reduced,  along 
the  vertical  and  horizontal  axes,  due  primarily  to  the  reduced  intensity  differences  along  the  portrait  edges. 

The  low-pass  spatial  filter  results  are  shown  in  Figure  3 1  ,b,3  for  f  =  4  c/pw  in  which  the  filtered  portrait  is  seen  to 
contain  the  general  form  information.  However,  the  general  homogeneous  contrast  of  the  features  of  the  face  found  in 
the  original  or  random  noise  portraits  is  lacking.  In  particular,  note  the  decreased  resolution  of  the  mouth  in 
comparison  to  the  other  (Hirtrait  configurations.  However,  even  with  this  portrait,  a  system  that  can  extract  similar 
form  information  appears  to  provide  sufficient  information  for  classification. 

I0.t>  Filtering  a  Binary  l.ine  Portrait 

( )ne  of  the  most  interesting  aspects  of  perception  is  that  line  patterns  can  represent  complex  forms  such  as  portraits. 
That  line  patterns  can  abstract  and  provide  basic  pattern  information  was  demonstrated  by  Attneave  ( Pllyl!.  He 
showed  that  information  about  forms  could  be  obtained  from  edges  or  contour  lines  of  a  form  at  points  of  change  in 
gradients  and  points  of  greatest  change  of  contour  direction.  But  how  do  edges  and  lines  contain  form  information-?  In 
general,  edges  and,  particularly,  lines  contain  high  spatial  frequency  information.  This  might  seem  to  be  at  odds  with 
the  notion  of  basic  form  information  being  provided  by  low  spatial  frequencies;  however,  it  has  been  demonstrated  (see 
Section  *M>I  that  the  low  spatial  frequencies  of  the  lines  of  the  similarity  groupings  contain  homogeneous  group 
information.  Perhaps  the  lines  of  complex  patterns  can  also  provide  form  information  in  the  low  spatial  frequencies.  If 
line  configurations  contain  low  frequency  information  which  is  similar  to  that  in  normal  grey-scale  portraits,  then 
perhaps  one  can  consider  lines  not  as  explicit  features  to  be  detected  and  manipulated  (therefore  explicitly  carrying  the 
form  information!  but  as  pattern  elements  that  can  provide  low  spatial  frequencies  that  have  been  shown  to  convey  the 
basic  form  information  of  objects.  The  subsequent  demonstrations  will  amplify  this  point. 

To  test  this  hy |M>thesis,  a  binary  line  portrait  was  created  from  an  edge  portrait  ( Figure  .32, a,  1 1  by  replacing  all  the 
picture  values  less  than  a  certain  value  by  1  (white!  and  those  greater  than  that  value  by  0  (black!,  as  shown  in  Figure 
3 1  .c,  1 .  Note  that  the  most  distinct  lines  come  from  regions  of  greatest  contrast  in  the  original  portrait. 

The  two-dimensional  log  magnitude  spectrum,  shown  in  Figure  31  ,c.2,  reveals  a  loss  of  elliptical  spatial  frequency 
structure  found  at  the  low  spatial  frequencies  of  the  other  portraits.  The  medium  spatial  frequencies  reveal  slight 
off-axis  ray  structure  in  comparison  to  the  original  portrait.  Although  the  low  frequency  energy  of  lines  is  quite  low 
compared  to  that  found  in  a  normal  portrait,  the  lines  still  provide  more  energy  at  the  low  spatial  frequencies  than  is 
found  at  the  high  frequencies. 

Istw-pass  spatial  filtering  to  f  =  4  c/f  demonstrates  basic  portrait  form  information  comparable  to  the  previous  portrait 
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configurations  (Figure  31,c,3l.  Note  the  hair,  cheek,  chin,  and  shoulder  outlines.  The  eyes,  nose,  and  mouth  are  the 
most  prominent  features  in  the  reconstruction.  Note  that  the  right  eye  is  the  most  emphasized  feature  and  that  the  nose 
and  mouth  cannot  be  resolved. 

It  apf>ears  that  the  low  spatial  frequency  forms  of  line  objects  are  similar  to  the  other  portrait  configurations.  It  is  worth 
noting  that  the  prominent  features  in  the  original  line  portrait  —  the  eyes,  nose,  and  mouth  —  that  are  most  dissimilar 
from  the  original  portrait  configuration  also  remain  most  dissimilar  in  the  low  spatial  frequency  reconstructed  line 
portrait.  This  result  suggests  that  even  complex  linear  binary  patterns  can  convey  certain  form  information,  from  only 
low  spatial  frequency  bands,  that  can  be  used  for  classification,  recognition,  and  identification.  Of  course,  in  these  and 
other  comparisons,  the  similarities  of  the  filtered  images  are  based  on  subjective  evaluation.  This  is  adequate  for  these 
studies.  More  quantitative  comparison  can  be  made  by  correlating  the  filtered  spectra  or  images  (e.g.,  see  Ginsburg, 
et  al..  l‘)7(>). 

10.7  Perception  of  Reversed  Contrast  Portraits 

The  preceding  filtered  line  portrait  revealed  similar  form  information  in  low  spatial  frequencies  when  compared  to 
other  portrait  configurations.  This  suggests  that  the  low  spatial  frequency  structure  of  lines  may  be  important 
information  for  perceived  similarity  of  objects.  However,  an  alternative  hypothesis  is  that  the  edge  information  of  the 
original  and  line  portraits  is  similar  and  thus  explicitly  provides  similar  portrait  information  (implying  that  the 
similar  low  spatial  frequency  information  is  coincidental).  Is  there  a  method  for  obtaining  the  same  edge  and  line 
information  while  varying  the  spatial  frequency  information?  The  answer  is  yes.  based  upon  a  consideration  of  the 
spatial  frequencies  of  the  negative  of  an  object.  A  reversed  contrast  object,  that  is.  one  with  black  and  white 
interchanged,  contains  identical  edge  information  but  dissimilar  spatial  frequency  information  (e.g.,  see  Lipson  and 
Lipson,  1%0,  Section  7.4.71.  If  the  edges  provide  the  basic  information  explicitly  (that  is,  identification  results  from 
the  detection  and  manipulation  of  the  edges),  then  it  would  be  expected  that  changing  a  black  edge  into  a  white  edge, 
and  vice  versa,  would  have  no  major  effect  on  the  perceived  portrait  information.  It  is  well  known  that  photograph 
negatives  are  harder  to  recognize  and  identify  than  positives,  a  result  that  immediately  suggests  that  edges  may  not 
carry  the  primary  information  for  form.  This  fact  is  demonstrated  by  two  positive  and  negative  portraits  shown  in 
Figure  32. a,  1,2. 

These  portraits  obviously  contain  similar  information.  Figure  32,a,2  has  reversed  picture  values  that,  although  they 
contain  identical  edge  information,  notably  affect  the  perception  of  the  eyes,  nose,  and  mouth  in  such  a  way  that 
portrait  similarity  and,  hence,  identification  are  decreased. 

A  more  interesting  example  of  the  inadequacy  of  isolating  edge  and  line  information  for  perception  is  shown  in  Figure 
32. b.  1 .2  where  the  original  grey-scale  portrait  was  converted  by  a  binary  programme  that  made  all  the  picture  values 
between  two  levels  white  and  all  other  values  black.  Again,  similarity  between  this  and  the  original  portrait  is  easily 
seen.  However,  when  these  values  are  reversed,  as  shown  in  Figure  32,b,2,  portrait  similarity  is  greatly  decreased. 
Except  from  critical  examination  of  the  head,  neck,  and  shoulder  outline,  portrait  information  is  very  dissimilar. 

Note,  in  particular,  that  even  with  much  effort,  the  mouths  of  the  two  portrait  configurations  do  not  convey  the  same 
information.  The  two  black  spots  are  highlights  in  the  lower  lip  of  the  positive  portrait  but  appear  as  two  teeth  in  the 
negative  portrait.  No  matter  how  conscious  an  attempt  is  made  to  isolate  these  features  from  the  influence  of  the 
surrounding  mouth,  segregation  resulting  in  another  interpretation  does  not  seem  possible.  These  effects  are  even  more 
apparent  when  the  portraits  are  viewed  at  about  2  degrees  of  visual  angle. 

These  results  further  demonstrate  the  classic  Gestalt  view  that  the  sum  of  the  parts  is  different  than  the  whole.  In  a 
filter  concept  of  perception,  highlights  cannot  be  extracted  explicitly  but  must  be  analyzed  as  a  whole  with  surrounding 
mouth  features.  These  results  also  suggest  that  it  is  the  low  spatial  frequency  information  that  carries  the  basic  portrait 
frequency  information.  By  viewing  the  portraits  from  a  distance,  the  high  spatial  frequencies  are  more  attenuated  than 
the  low  spatial  frequencies,  and  the  relative  dissimilarities  between  the  portrait  configurations  are  seen  to  remain. 


10.8  Summary 

This  chapter  extended  the  application  of  the  concept  of  filtering  from  simple  to  complex  objects:  portraits.  Subjectively 


similar  form  information  was  shown  to  exist  within  a  two-octave  band  of  low  spatial  frequencies  for  a  wide  variety  of 
different  portrait  configurations,  including  noise  and  line  versions.  Visual  texture  in  terms  of  spatial  filtering  is 
investigated  next. 
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CHAPTER  11 


MSI  At  textures 


1 1.1  Introduction 

Our  visual  world  is  rich  in  textures — multiple  patterns  that  share  some  common  spatial  properties.  Understanding  how 
the  visual  system  extracts  information  from  texture  is  important  because  of  the  role  that  texture  plays  in  separating 
objects  from  their  background  I  figure-ground  I.  e.g.,  Julesz  1 19711.  Furthermore,  this  understanding  is  necessary  if 
machines  are  to  be  made  to  perform  similar  functions  iGinsburg.  19731. 

Textures  can  be  defined  in  terms  of  either  spatial  features  or  transformations.  The  explicit  extraction  of  spatial 
features,  such  as  edges,  lines,  orientation,  contrast,  etc.,  has  been  suggested  by  many  researchers,  e.g.,  Hawkins 
( 19701.  This  section  shows  how  much  simpler  and  parsimonious  filtering  can  be  in  extracting  certain  textures  when 
used  in  conjunction  with  operations  consistent  with  known  visual  processes. 

Spatial  frequency  content  has  been  shown  to  distinctly  characterize  certain  textures,  e.g.,  Cutrona  1 19651;  Lendaris  and 
Stanley  1 19701.  For  example,  certain  geographical  characteristics  can  be  distinguished  by  their  spatial  frequency 
content,  e.g..  Goldstein  and  Rosenfeld  <  19641.  Although  substantial  overlap  can  occur  in  spectra  from  certain  complex 
scenes  (Johnston.  El-Sum,  Rudin,  Sikorsky,  and  Feigenbaum.  19641,  there  still  can  be  good  separation  in  certain 
regions  of  spatial  frequencies,  (see  Figure  4  of  Hawkins,  19701.  The  search  for  different  spectral  bands  to  separate 
various  textures  may  prove  more  fruitful  than  an  analysis  of  the  complete  spectrum,  just  as  generalized  forms  were 
shown  to  exist  from  only  the  low  spatial  frequencies  of  objects  when  the  rest  of  the  spectrum  was  filtered  out. 

1 1 .2  Two  General  Figure-Ground  Problems 

It  was  shown  in  Sec  tion  9.6  that  low -pass  filtering  can  offer  an  explanation  for  the  perception  of  figure-ground  from 
textures  made  of  dots  and  lines.  Here,  an  initial  investigation  of  figure-ground  is  made  in  terms  of  the  anisotropic  filter 
characteristics  and  the  spatial  frequency  /orientation-selective  nature  of  channels  in  the  visual  system. 

1 1 .2. 1  Isolating  a  Rectangular  Form  from  Oblique  Lines  by  Spatial  Filtering 

Consider  the  figure-ground  object  in  Figure  33, a.  The  problem  here  is  to  extract  the  rectangular  form  from  the 
background.  This  is  a  simple  task  for  an  anisotropic  spatial  filter,  such  as  the  MTF(H  I.  that  attenuates  the  oblique 
structure  in  Figure  33. b.  Similar  extraction  of  the  rectangle  could  be  achieved  if  the  spectral  components  in  the  obliques 
were  removed.  Note  that  low-pass  filtering  ( Figure  33,c,d.  I  only  signals  that  there  is  an  object  in  the  original  pattern. 

1 1 .2.2  Isolating  Oblique  Structure  in  a  Real  Scene  by  Spatial  Filtering 

Another  example  of  the  power  of  spatial  filtering  that  can  enhance  specific  regions  of  spatial  frequency  in  a  global 
manner  is  shown  in  Figure  34.  Note  that  all  the  collinear  lines  in  this  real  scene  can  be  attended  to  in  one  orientation  at  a 
time.  Filtering  out  the  two  different  oblique  regions  of  the  spectrum  separates  the  linear  objects  into  two  groups,  as 
shown  in  Figure  34, b  and  c. 

It  should  be  clear,  once  any  object  is  analyzed  into  quasi-independent  channels  of  different  spatial  frequencies  (sizel  and 
orientation,  that  such  isolation  of  objects  is  quite  simple.  However,  does  the  filter  concept  have  predictive  |M»wer  with 
other  figure-ground  problems  such  as  letter  arrays,  random  dot  patterns,  and  more  complex  scenes?  Furthermore,  how 
can  those  objects  that  are  filtered  be  separated  from  the  field?  These  questions  are  examined  next. 

1 1 .3  Similarity  Grouping  of  Letter  Arrays 

The  variables  of  similarity  grouping  found  important  by  Julesz  ( 19621.  Beck  ( 1 966, a. b;  1967;  19721.  and  Olson  and 
Attneave  ( 19701  are  shape,  slope,  and  brightness.  The  results  of  Pickett  (19701  agree  with  those  findings  and  also  with 


the  results  of  Julesz  ( 19711  which  suggest  that  the  visual  system  contains  a  "slicer"  mechanism  that  operates  on  texture 
clusters  of  similar  brightness.  Ginsburg  1 197 1, a. b;  1973),  and  Ginsburg,  et  al.  (19761,  have  shown  that  anisotropic 
low-pass  filtering  can  segregate  certain  textures  into  objects  of  homogeneous  brightness.  Here,  similar  groupings  using 
the  MTF  filter  are  shown. 

1 1 .3. 1  Similarity  Grouping  Due  to  Orientation  Differences 

Texture  fields  that  contain  repeated  objects  differing  only  in  orientation  will  not  be  considered  because,  as  previous 
examples  of  orientation-selective  filtering  have  shown,  they  can  be  easily  separated  from  one  another  by 
quasi-independent  orientation  channels. 

One  of  the  major  problems  with  present  theories  of  feature  analysis  is  typified  by  a  basic  dichotomy  that  emerges  from 
the  results  of  Beck  ( 1966, a, b|  and  Eichelman  ( 19701.  Similarity  judgements  of  paired  letters  had  little  predictive  power 
in  determining  how  the  letters  would  group  when  they  were  made  into  arrays.  An  example  is  shown  in  Figure  33, a,  in 
which  the  tilted  T  ITT)  may  be  seen  as  more  similar  to  the  upright  T  lUT)  than  is  the  L.  However,  when  these  same 
letters  are  arranged  in  arrays  (Figure  35, b)  the  opposite  grouping  occurs. 

Ginsburg,  et  al.  11976),  have  suggested  that  the  paired  letters  are  judged  for  similarity  with  the  same  metric  used  for 
machine  pattern  classification  tasks — Euclidean  distance  (cross-correlation)  of  the  low  spatial  frequencies  of  the  letters. 
However,  the  arrays  of  letters  appear  to  be  processed  in  a  global  manner.  Comparing  the  features  of  the  individual 
letters  would  provide  the  same  grouping  as  paired  letters.  However,  the  letter  arrays  can  be  segregated  by  a  more  global 
property,  e.g..  brightness  distribution  due  to  the  anisotropic  spatial  filtering  of  the  visual  system  (Ginsburg,  et  al., 

1976).  Note  that  the  cross-correlation  is  a  measure  of  the  similarity  of  the  letters,  whereas  the  effects  of  brightness 
provide  measurable  differences  between  the  letters. 

The  segregation  of  the  letters  in  letter  arrays  by  the  MTF  filter  is  shown  in  Figure  36.  The  TTs  are  clearly  less  visible 
than  the  UTs  and  Ls  I  Figure  36, c).  (These  results  are  general  demonstrations  of  the  effects  of  the  MTF  filtering 
because  the  coarse  quantization  of  the  TTs  in  these  pictures  provides  additional  information  by  which  the  letters  can  be 
segregated.) 

The  effects  of  the  MTF  filter  are  particularly  relevant  to  the  objects  used  by  Beck  ( 1966, a, b;  1967;  1972).  The  size  of 
the  objects  ranged  from  only  10  to  3 1  minutes  and  luminance  values  were  scotopic,  0.023  to  0.29  foot-lamberts.  These 
values  suggest  that  similarity  judgements  were  being  made  with  high  spatial  frequency  patterns  of  low  contrast.  From 
the  discussion  in  Section  5.2. 1 ,  these  are  the  conditions  that  produce  the  greatest  degree  of  anisotropy  in  the  human 
visual  system.  Experimental  predictions  based  on  this  analysis  are  discussed  in  the  next  section. 

1 1 .3.2  Similarity  Grouping  Due  to  Configuration  Differences 

Beck  1 1966. b)  used  simple  geometric  objects  and  letters  to  determine  the  effect  of  orientation  and  shape  similarity  on 
|>creeptual  grouping  by  having  subjects  partition  arrays  of  different  patterns  into  two  regions.  Studies  by  the  author 
w  ith  anisotropic  filtering  readily  segregated  the  patterns  in  terms  of  orientation.  Independent  orientation  channels 
easily  handle  such  tasks.  However,  some  of  his  arrays  contained  objects  with  only  shape,  not  orientation  differences. 

( ian  these  results  also  be  explained  by  brightness  differences  due  to  the  MTF  filtering?  The  answer  appears  to  la*  yes. 
based  on  the  results  of  the  MTF  filter  on  a  typical  array  in  Figure  37.  Here  the  Ls  segregate  from  UTs  and  sideways 
TslST)  by  brightness  also  because  of  the  low-pass  filtering  characteristics  of  the  MTF. 

The  magnitude  spectra  for  both  the  T  and  l,  in  Figure  37,  normalized  from  0  to  10.  are  shown  in  Figure  38.  (The 
spectrum  of  the  sideways  T  is  not  shown  because  it  is  the  same  as  the  UT  rotated  by  'Ml  degrees.  I  Note  that  1,  has  less 
sfieetral  energy  at  the  low  spatial  frequencies  than  does  the  T  (see  the  squares  in  Figure  381.  Thus,  it  is  not  too 
surprising  that  the  I,  will  be  affected  more  by  the  low-pass  filtering  characteristics  of  the  MTF  than  the  T. 

I  he  numerical  printouts  of  the  filtered  array  I  Figure  39, a. b I  confirm  the  observations  with  the  pictures.  Also  note  that 
the  intensity  distribution  of  the  same  letters  is  not  equal.  This  result  suggests  that  grouping  of  such  objects  depends  on 


some  local  average  brightness,  not  on  precise  values.  Furthermore,  this  example  suggests  that  changes  in  the  brightness 
distribution  occur  with  different  pattern  configurations.  Thus,  the  proximity  of  similar  patterns  affects  judgements 
based  on  local  brightness.  The  magnitude  of  such  effects  on  similarity  judgements  must  be  determined  experimentally. 

Many  of  the  other  results  of  Beck  and  Olson  and  Attneave  were  analyzed  in  a  similar  manner,  but  these  results  would 
require  a  lengthy  discussion.  The  effects  of  low-pass  filtering  on  grouping  of  these  kinds  of  arrays  can  be  demonstrated 
easily  by  squinting  or  viewing  the  arrays  (Figure  40)  at  a  distance.  Different  brightness  distributions  occur  as  Figure  40 
is  rotated  to  different  orientations.  In  general,  it  appears  that  not  only  the  primary  results  of  the  experiments  of 
Beck  and  Olson  and  Attneave  can  be  explained  in  terms  of  filtering  but  also  the  second  and  third  order  effects. 

1 1 .3.3  Second  Order  Effects  of  Grouping:  Symmetry 

One  example  of  a  second  order  effect  will  be  discussed.  Beck  ( 1006, a  I  reported  in  two  experiments  that  the  TT  grouped 
further  away  from  the  UT  than  the  X.  These  results  cannot  be  explained  by  differences  in  orientation  because  the  TT 
and  X  have  the  same  orientation.  However,  the  X  has  two-fold  symmeti  y;  the  'IT  does  not.  Thus,  the  X  will  have  more 
energy  in  the  lowest  spatial  frequencies  than  will  the  TT.  This  may  be  s<  en  by  comparing  the  numerical  printouts  of  the 
normalized  intensity  values  of  the  filtered  X  and  IT  shown  in  Figure  41  ,a.b.  The  total  intensity  for  values  of  the 
images  equal  to  or  greater  than  30  for  the  X  is  418  and  only  314  for  the  TT.  Thus  the  TT  contains  about  23  percent 
of  the  intensity  of  the  X  for  values  of  intensity  equal  to  or  greater  than  30. 

Noting  the  intensity  distributions  of  the  MTF  filtered  UT  and  X  and  UT  and  TT  (Figure  41  ,a.bl.  it  is  quite  evident  that 
the  X  would  provide  a  brighter  object  than  the  TT.  This  result  helps  to  explain  why  subjects  reduced  the  brightness  of 
the  UTs  in  an  array  of  UTs  and  Xs  less  than  in  an  array  of  UTs  and  TTs.  There  is  a  greater  difference  between  the 
brightness  <if  the  UTand  TT  than  the  UT  and  X. 

The  filter  analysis  is  consistent  with  the  information  about  the  intensity  of  the  letters  that  were  presented  to  the  subjects 
in  the  experiments.  The  array  sizes  used  by  Beck  ( 1 066, a  I  ranged  from  3  degrees,  10  minutes  to  about  10  degrees.  30 
minutes.  As  Beck  (10721  pointed  out,  peripheral  acuity  blurred  peripheral  objects.  Beck  used  that  fact  to  suggest  that 
the  poor  grouping  of  the  Xs  occurred  because  they  would  readily  lose  discriminability  with  blur.  However,  the  major 
difference  between  the  filtered  letters  ( Figure  4 1  ,b  I  is  the  increased  intensity  at  the  centre  of  the  X  compared  to  the  TT. 
Although  the  overall  extent  of  the  X  is  slightly  reduced  in  comparison  to  the  TT  after  filtering  (as  these  filtered  letters 
show),  the  central  intensity  values  of  these  letters  suggest  that  the  X  will  retain  more  form  information  than  the  TT. 

Beck  further  supports  his  argument  with  his  finding  that  when  the  Xs  were  larger,  they  grouped  as  well  as  the  tilted  Ts 
(Beck.  1*172,  p.  16).  However,  the  size  of  the  letters  was  changed  by  only  a  factor  of  1 .2  I.')  mm  to  6  mm)  in  the  two 
experiments  that  he  discusses.  The  mean  luminance  of  the  standard  pattern  in  the  two  experiments  differed  only  by  a 
factor  of  2.3  (0. 1 14  to  0.2*)  foot-lambertsl.  Finally,  his  statistical  analysis  showed  no  significant  difference  between  the 
grouping  strength  of  the  X  and  TT  in  the  second  experiment;  the  difference  between  the  mean  luminance  values 
required  for  grouping  in  the  two  different  experiments  were  quite  similar,  0.013  and  0.013  foot-lamberts.  This  analy  sis 
of  his  results  suggests  that  there  was  little  if  any  difference  between  the  grouping  of  the  X  and  TT  in  the  two 
experiments — a  result  that  is  consistent  with  filtering.  Other  filtering  by  the  author  suggests  that  small  changes  in  size 
(a  factor  of  1.2)  would  produce  too  little  difference  in  the  intensity  distribution  of  these  objects  to  change  the  general 
results.  This  analysis  further  suggests  that  it  is  the  relative  brightness,  i.e.,  contrast,  between  the  objects  due  to  filtering 
and  not  the  relative  discriminability  that  is  the  major  grouping  variable  in  these  experiments.  ( )f  course,  further 
investigation  is  required  to  assess  the  relative  importance  of  each  of  these  variables. 

These  filtered  letters  suggest  a  counter-intuitive  prediction— The  similarity  judgements  are  based  primarily  on  the 
brightness  differences  of  the  different  letters  due  to  the  filtering  effects  on  shape  and  orientation.  T  his  suggestion 
predicts  that  increasing  the  brightness  of  tilted  objects  in  an  array  will  reduce  their  discriminability  under  certain 
conditions.  Experiments  support  this  prediction  ( Eichelman  and  Ginsburg,  I *>77). 

To  summarize,  arrays  of  patterns  such  as  those  used  by  Beck  ( 1*166, a. b;  1%7;  1*172)  and  Olson  and  Attneave  1 10701 
appear  to  group  in  a  manner  consistent  with  those  predicted  by  brightness  effects  based  on  anisotropic  spatial  filtering 
of  the  slope  and  shape  of  the  patterns.  These  array  s  are  made  up  of  relatively  simple  patterns.  Is  the  filler  concept 


relevant  to  more  complex  figure-ground  patterns  such  as  random  dot  patterns?  These  are  investigated  in  the  next 
section. 


11.4  Isolating  Objects  from  Random  Dots 

11.4.1  Isolating  Form  and  Symmetry  from  a  Random  Dot  Pattern 

Perhaps  some  of  the  most  complicated  figure-ground  problems  involve  the  extraction  of  objects  from  random  patterns. 
An  example  is  given  in  Figure42,a — a  random  dot  pattern  created  by  Julesz  11971 ).  The  problem  here  is  to  understand 
how  the  visual  system  so  rapidly  and  seemingly  effortlessly  extracts  symmetry  information  and  the  face-like  objects 
from  such  a  noisy  pattern.  Julesz  (1971,  19751  suggested  that  textures  differing  in  first  order  statistics  (e.g.,  intensity 
values)  are  easily  seen.  However,  he  does  not  provide  any  insight  into  how  the  visual  system  performs  such  tasks.  Marr 
( 19791  also  suggested  that  first  order  discrimination  and  grouping  processes  were  used  to  extract  forms  from  filtered 
textures.  Unfortuantely,  Marr's  complex  techniques  were  based  upon  ad  and  post  hoc  assumptions  of  what  makes  up 
and  extracts  information  from  objects.  Ginsburg  ( 1 973 1  showed  that  the  filter  characteristics  of  the  visual  system — the 
two-dimensional  MTF — can  create  somewhat  homogeneous  forms  whose  first  order  statistics  (intensity )  were  used  to 
begin  isolation  and  extraction  of  form  from  a  random  dot  pattern. 

The  result  of  that  spatial  filtering  is  shown  in  Figure  42.  The  original  pattern  is  filtered  by  the  MTF  (Figure  42, bl  or  a 
low-pass  (24  by  241  filter  ( Figure  42, c)  to  cluster  proximate  dots  of  similar  intensity.  This  result  demonstrates  that 
tfeneral  filtering  processes  in  the  visual  system  can  form  homogeneous  clusters  in  a  simple  manner.  Furthermore,  when 
retinal  inhomogeneity  is  considered,  such  filtered  information  has  to  exist  in  at  least  the  peripheral  retina  when  the 
original  pattern  is  viewed.  The  general  symmetry  of  this  complex  pattern  can  be  found  easily  from  the  low  spatial 
frequencies  of  the  pattern,  as  shown  in  Figure  41, d,e.  It  is  suggested  that  higher-order  visual  processes  can  attend  to 
symmetry  in  objects  filtered  from  channels  whose  bandwidth  captures  relatively  low  spatial  frequencies.  More  detail 
about  this  point  is  provided  in  Section  19.4.9. 

1 1 .4.2  Isolation  Objects  of  Different  Size  in  a  Random  Dot  Pattern 

Another  example  of  how  a  filtered  image  separates  figure  from  ground  by  creating  different  areas  of  intensity  from 
similar  random  distributions  of  different  size  objects  is  shown  in  Figure  43.  Here,  the  small  square  can  be  seen 
Immediately  in  the  lower  right  corner.  The  region  of  small  squares  is  seen  to  have  different  average  contrast  than  the 
rest  of  the  picture  after  MTF  filtering.  The  high  frequency  attenuation  characteristics  of  the  MTF  filter  reduce  the 
average  contrast  of  the  region  of  small  squares.  The  binary  picture  in  Figure  43, c  is  created  from  the  upper  30  percent 
of  the  intensity  values  of  Figure  43, b  to  show  the  differences  between  the  peak  intensities  inside  and  outside  the  square 
region. 

11.5  Analysis  of  a  Complex  Scene 

Results  of  previous  research  into  texture  analysis  (Ginsburg,  19731 — some  of  which  were  presented  here — were  used  to 
analy  ze  the  complex  scene  shown  in  f  igure  44, a.  The  problem  is  the  identification  of  letters  in  the  words  by  using 
filtering  methods  that  may  be  similar  to  those  used  by  the  visual  system.  Figure  44. b  represents  the  selected  area  of 
interest  and  the  general  amount  of  information  that  may  be  contained  within  the  parafoveal  region.  A  log  transform  of 
the  picture  in  Figure  44. b,  shown  in  Figure  44. c,  improves  the  contrast  of  the  features  of  the  letters.  Figure  44. d  shows 
the  result  of  low-pass  filtering.  The  lack  of  general  form  information  demonstrates  that  a  global  analysis  based  on 
filtered  images  of  this  kind  would  be  unsuccessful  in  providing  sufficient  information  for  the  identification  of  these 
letters. 

How  can  letters  be  isolated  and  recognized  on  such  a  noisy  background?  Previous  results  on  texture  analysis  suggested 
that  objects  could  be  extracted  from  differences  in  brightness.  F  irst,  the  picture  is  digitized  into  128  by  128  pels  for 
computer  processing.  This  is  a  crude  method  of  low-pass  filtering.  Digitally  slicing  approximately  30  percent  of  the 
lowest  intensities  contained  in  the  picture  (shown  as  the  brightest  objects  in  F’igure  44, el  and  using  them  to  create  the 
picture  in  F  igure  44, f  captures  the  elements  that  make  up  each  letter,  plus  a  small  amount  of  noise.  ( A  ±  5  percent 
change  in  the  intensity  values  that  were  selected  works  equally  well.  I  The  solution  to  the  remaining  problem  of 
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classifying  the  letters  from  these  discrete  features  has  already  been  offered  for  man  and  machine  (Ginsburg,  1971  .a. bl. 
The  "Gestalt”  of  these  objects  is  found  from  the  low  spatial  frequencies,  as  seen  from  Figure  44, g  using  the  MTF  filter 
or  Figure  44, h  using  ideal  low-pass  filtering.  In  practice,  each  letter  would  be  isolated  and  processed  separately  as  done 
by  the  fovea  when  these  pictures  are  held  at  normal  reading  distance. 

There  are,  of  course,  many  other  possible  ways  to  extract  similar  information  from  the  original  picture.  The  point  here 
is  that  it  has  been  accomplished  using  simple  methods  believed  to  correspond  to  the  filtering  properties  of  the  human 
visual  system.  Furthermore,  these  results  have  gone  beyond  simply  the  extraction  of  the  relevant  features.  The 
information  that  was  extracted  is  sufficient  for  classification  by  man  and  machine  (Ginsburg,  1973). 

1 1 .6  Monocular  Texture  Gradients 

Certain  lighting  conditions  can  create  shadows  (contrast  gradients)  that  cause  certain  patterns  to  be  perceived  in  depth. 
Gibson  ( 1950)  has  shown  that  certain  textures  whose  elements  have  the  same  contrast  and  size  and  converge  at  some 
point  can  also  convey  a  perception  of  depth  with  monocular  viewing  (Figure  45. a).  Can  any  relationship  be  shown  to 
exist  between  contrast  gradients  of  the  filtered  image  and  of  similar  textures  and  the  perception  of  depth?  The  author 
noted  that  pattern  elements  that  were  close  to  one  another  or  were  of  different  size  generally  had  different  contrast 
after  filtering.  These  results  prompted  filtering  of  a  digitized  textured  field  taken  from  Gibson  and  shown  in  Figure 
45, b.  Note  that  the  pattern  elements  originally  having  the  same  contrast  but  different  sizes  and  proximity  give  the 
impression  of  perceived  depth.  The  MTF  filtered  image  (Figure  45. cl  shows  that  the  pattern  elements  of  different  sizes 
and  proximity  havp  different  contrast.  It  is  possible  that  one  aspect  of  perceived  monocular  depth  is  the  different 
relative  intensity  levels  (contrast)  of  the  features  in  the  filtered  image  of  a  channel  (See  Section  13.6  for  other  examples.  I 

11.7  Summary 

This  chapter  provided  demonstrations  of  techniques  based  on  suggested  processing  capabilities  of  the  visual  system 
which  attempt  to  solve  problems  of  the  perception  of  visual  textures.  The  important  grouping  variables  that  other 
researchers  have  found— object  shape,  orientation,  and  brightmess — were  shown  to  be  provided  by  filtering 
mechanisms  in  the  visual  system.  The  success  of  a  counter-intuitive  experiment  based  on  these  results  suggested  the 
predictive  power  of  these  techniques  and  analyses.  A  complex  scene  was  analyzed  for  the  extraction  of  objects  sufficient 
for  machine  classification  using  filtering  methods  based  on  the  previous  analysis.  Finally,  an  example  of  a  texture 
gradient  that  provides  monocular  depth  cues  was  examined  by  spatial  filtering.  Next,  the  same  spatial  filtering  is 
applied  to  offer  an  explanation  for  the  perception  of  visual  illusions. 
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CHAPTER  12 


VISUAL  ILLUSIONS 


12.1  Introduction 

An  understanding  of  visual  illusions  is  important  because  they  demonstrate  instances  in  which  the  visual  system  fails 
seemingly  simple  tasks.  Lines  and  angles  in  very  simple  geonn  trie  patterns  are  seen  distorted  into  different  shapes, 
lengths,  and  orientations.  A  full  understanding  of  perception  would  have  to  include  an  explanation  of  visual  illusions. 

A  number  of  theories  of  visual  illusions  have  been  proposed.  Contemporary  theories  to  account  for  the  appearance  of 
illusions  have  invoked  such  factors  as  distortions  due  to  diffraction  and  optical  aberrations  in  the  eye,  (e.g..  Chiang. 
19681,  physiological  mechanisms,  e.g..  lateral  inhibition  (e.g.,  Ganz  and  Day.  1965;  von  Bekesy,  1968;  Blakemore. 
Carpenter,  and  Georgeson,  19701.  and  cognitive  explanations  (e.g.,  Gregory,  1963;  Day,  19721.  The  reader  is  referred 
to  an  excellent  review  by  Robinson  ( 19721  on  illusions  and  the  various  theories  used  to  attempt  to  explain  them. 

The  analysis  of  visual  illusions  presented  here  represents  a  continuation  of  research  by  the  author  that  regards  illusions 
as  a  consequence  of  the  manner  in  which  the  visual  system  processes  all  spatial  objects:  by  spatial  filtering  (Ginsburg. 
1971, a, b;  1973;  1975;  Ginsburg.  etal.,  19761. 

In  the  introduction  of  this  dissertation,  it  was  stressed  that  the  visual  system  must  be  throwing  away  large  amounts  of 
detailed  information.  In  Chapters  5,  6,  and  7,  the  neurophysiological  and  psychophysical  evidence  for  the  limits  of 
spatial  resolution  in  the  fovea  as  well  as  the  parafovea  was  discussed.  Furthermore,  the  spatial  bandwidth  of 
mechanisms  farther  up  the  visual  pathway  becomes  increasingly  narrow. 

There  must  be  some  consequence  to  the  throwing  away  of  certain  details  about  objects  at  the  various  stages  of  visual 
processing.  Two  consequences  of  losing  detail  are  that  intensity  and  spatial  information  are  distorted,  as  has  been  shown 
previously  in  many  of  the  filtered  images.  From  this  point  of  view,  visual  illusions  result  primarily  from  the  spatial 
filtering  properties  of  the  visual  system.  Thus,  filtering  geometric  and  contrast  illusions  with  spatial  filters  bast'd  on  the 
filtering  characteristics  of  the  visual  system  generally  resulted  in  the  distortions  that  are  perceived  when  they  are 
viewed,  as  shall  now  be  shown. 

12.2  Illusions  of  Noncollinearity 

12.2. 1  Variations  and  a  "Correction"  of  the  I'oggendorff  Illusion 

In  the  Poggendorff  illusion  (Zbllner,  18621,  the  collinear  oblique  lines  that  transverse  the  parallel  lines  are  not  perceived 
as  collinear  I  Figure  46, a,  1 1.  Since  this  illusion  depends  upon  judgement  made  on  the  collinearity  of  lines  that  form  an 
angle  with  other  lines,  the  spatial  filtering  results  >n  the  Miiller-Lyer  illusion  (Ginsburg,  1971, a. b;  Ginsburg.  et  al.. 

19761  would  predict  a  shift  in  the  intensity  distribution  at  the  intersection  of  the  collinear  oblique  lines  and  the  parallel 
lines  into  the  acute  angle. 

I'he  result  of  VI TF  filtering  and  low-pass  filtering  with  and  without  MTF  prefiltering  is  shown  in  Figure  46. a, 3. 4. 5. 
Placing  a  straight  edge  over  the  peaks  of  the  intensity  distribution  of  the  collinear  lines  demonstrates  the  lack  of 
collinearity.  especially  in  the  low-pass  filtered  images.  However,  even  though  the  filtered  objects  have  been  shown  to 
well  represent  the  intensity  distribution  of  the  computer  plots  and  printouts,  the  numerical  printouts  shown  in  Figure 
4 7,a,b  provide  objective  quantitative  measures  of  the  distortions  due  to  spatial  filtering.  The  filtered  images  and  the 
numerical  printouts  confirm  the  prediction  and  suggest  that  the  illusion  is  due  to  bending  of  collinear  lines  at  their 
junction  with  the  parallel  lines. 

In  this  filtered  image  and  those  that  follow,  the  magnitude  of  the  distortions  is  assumed  to  be  determined  by  the 
following  rules.  The  orientation  of  a  line  is  judged  on  the  basis  of  the  axis  connecting  the  two  highest  intensity  peaks 
after  filtering.  The  length  of  a  line  is  judged  on  the  basis  of  distance  from  the  peak-to-peak  intensity  values  of  the  line 


and/or  the  distance  from  intensity  edge  to  intensity  edge,  depending  u|H>n  which  measure  is  easier  to  determine  in  these 
demonstrations.  An  exfieriment  discussed  in  Section  1 2.8. 1  attempted  to  determine  the  relevant  intensitv  features  of 
the  filtered  images  of  an  illusion. 

If  similar  spatial  filtering  in  the  \  isual  system  accounts  for  the  distortion  in  the  Poggendorff  illusion,  then  an  inverse 
M  1  T  filter  should  provide  the  intensity  distribution  that  cancels  the  distortion.  An  inverse  filter.  MTF'  I .  filtered  the 
original  illusion  with  the  general  result  that  the  collinear  lines  were  shifted  in  the  opjiosite  direction  at  the  junctions  with 
the  parallel  lines  (Figure  47. cl. 

It  is  emphasized,  however,  that  this  inverse  filter  technique  is  only  an  approximation  becuuse  the  minimum  MTF 
values  become  infinitely  large,  and  v  ice  versa,  when  the  original  values  of  the  MTF  are  inverted.  This  is  why  the 
parallel  lines  are  almost  absent  in  the  M  IT '  I  numerical  printout:  they  are  greatly  reduced  in  the  MTF’  I  filtered  image 
in  comparison  to  the  obliques.  I  urthermore.  the  inverse  filter  technique  api>ears  useful  only  for  geometric  illusions  that 
involve  angles  and  not  absolute  line  length  su<  h  as  the  horizontal-vertical  line  illusion  of  Figure  7)8. a.  Other  studies  by 
the  author  suggest  that  the  inverse  filter  cannot  distribute  intensities  to  places  where  none  existed  in  the  original  object: 
it  ap|>ears  to  be  only  able  to  shift  existing  intensity  distributions. 

1'he  original  results  of  the  filtering  suggest  that  the  "correction"  to  this  configuration  of  the  I’oggendorff  illusion  is  to 
bend  the  collinear  lines  at  the  junction  opposite  the  filtered  illusion.  This  correction  * .<  shown  in  Figure  48. b  w  here,  in 
comparison  to  the  standard  Poggendorff  illusion  ( Figure  48. a I.  the  corrected  illusion  shows  a  substantial  reduction  in. 
if  not  a  complete  loss  of.  the  perceived  distortion.  The  reader  is  encouraged  to  place  a  straight  edge  across  the  collinear 
lines  to  see  the  small  degree  of  curvature  at  the  junction  that  is  present. 

It  is  interesting  to  note  that  Helmholtz  I  l‘HI<)|  suggested  that  the  original  Poggendorff  illusion  was  due  to  the  bending 
of  collinear  lines  at  the  junction  where  the  obliques  meet  the  parallel  lines,  similar  to  that  found  in  the  numerical 
printouts  of  the  MTF  filtered  illusion. 

koboyashi  1 1  *).")(> I  first  demonstrated  that  the  Poggendorff  illusion  disap|ieared  when  the  obliques  were  bent  at  the 
intersection  with  the  parallel  lines.  The  filtered  data  presented  here  show  why  such  a  demonstration  works. 

The  corrected  illusion  does  not  seem  to  be  greatly  dependent  upon  viewing  distance.  An  interesting  fact  is  observed  at 
about  0.:»  degrees:  The  corrected  illusion  exhibits  negligible  distortion  even  though  the  correction  is  not  seen  at  that 
distance.  Also  note  that  the  illusion  persists  lalthough  somewhat  decreased  in  comparison  to  the  original  illusion  I  when 
the  intersections  of  the  oblique  lines  with  the  parallel  lines  are  removed  I  Figure  48. cl.  It  should  be  clear  from  the 
earlier  examples  of  filtered  images  that  low-pass  filtering  this  particular  illusion  will  still  cause  a  shift  in  the  intensity 
distribution  in  the  same  direction  as  perceived  distortion.  These  two  observations  las  well  as  other  arguments  presented 
laterl  suggest  that  perceived  distortions  are  based  upon  spatial  information  present  in  low  spatial  frequency  channels. 

Two  variations  of  the  Poggendorff  illusion  I  Pressey  and  den  II ever,  1*M>8I  are  shown  in  Figure  48. b.  1  ,e,  I .  The  original 
Poggendorff  illusion  persists  with  the  oblique  angles  only  (Figure  4<>.b.  1  I.  However,  the  illusion  is  diminished  (Pressey 
and  <len  (lexer.  FK>8l  or  reversed  I  Pestle.  I%<)|  when  only  the  acute  angles  of  the  Poggendorff  illusion  are  present 
( Figure  4f».e.  1 1.  In  this  case,  if  judgements  are  made  of  the  vertical  position  of  the  obliques  bast'd  on  the  peak  intensities 
or  extremities  of  the  filtered  objects,  a  continuum  of  judgements  can  be  made  which  result  in  a  decreased  or  reversed 
illusion. 

There  are  other  attributes  of  the  filtered  illusion  such  as  small  changes  in  angle  and  curvature  of  the  collinear  lines 
whose  relative  inqiorlance  to  the  illusion  appears  to  be  second  order  and  will  not  lie  discussed  further  here. 

12.2.2  An  Illusion  Due  to  Krror  in  Aligning  Kdges 

A  very  simple  geometric  illusion,  shown  in  Figure  4'l.a.  was  rejiorted  by  Isieb  ( IH'l.'tl.  The  short  line  at  the  right  is  seen 
slightly  above  the  longer  liottom  line  at  the  left  with  which  it  is  collinear.  This  illusion  is  stronger  when  it  is  rotated  4.’i 
degrees  I  Ilotopf.  I I.  The  M  TF  filtered  image  shows  the  |>erceived  distortions  in  the  numerical  printout  of  Figure 
4't.b.  Note  the  relative  shift  of  the  average  intensity  on  each  side  of  the  original  line.  The  parallel  lines  on  the  left  shift 


away  from  each  other.  The  line  on  the  right  remains  in  essentially  the  same  position.  The  overall  effect  is  that  the  line 
on  the  right  appears  to  be  displaced  upwards.  These  effects  are  increased  when  the  illusion  is  rotated  45  degrees  (not 
shown),  due  to  the  anisotropy  of  the  two-dimensional  MTF. 

12.5  Illusions  of  Line  Length 

12.3. 1  Variations  of  the  Miiller-Lyer  Illusion 

A  digitized  representation  of  the  MUller-Lyer  illusion  1 1889)  is  shown  in  Figure  50, a,  1 .  Lines  of  equal  length  are 
perceived  to  be  longer  and  shorter  if  out-pointing  and  in-pointing  fins  are  added  to  the  ends  of  the  lines. 

A  variation  of  the  Miiller-Lyer  illusion  has  the  fins  only  (Brentano,  1892),  as  shown  in  Figure  50,b,l.  Here  the  illusion 
is  that  the  distance  between  the  tips  of  the  out-going  fins  is  less  than  the  distance  between  the  tips  of  the  in-going  fins. 

The  MTF  (Figure  50.  a,  b.  3 1  and  low-pass  filtered  images  of  both  illusions  (Figure  50, a, b, 4. 5)  show  the  perceived 
distortions,  confirming  the  optical  filtering  results  of  Ginsburg  ( 1971, a, b). 

Miiller-Lyer  ( 1896)  created  a  variant  of  his  original  illusion,  a  digitized  version  of  which  is  shown  in  Figure  50. c.  1 .  Here 
the  extension  of  the  fins  causes  a  perceived  decrease  in  line  length  compared  to  the  same  line  with  shorter  fins. 

This  illusion  provides  one  counter-example  against  a  general  optical  blur,  smoothing,  or  assimilation  theory  for 
geometric  illusions.  Any  averaging  process  would  keep  the  lines  the  same  length  or  increase  the  length  of  the  line  with 
the  longer  fins  because  there  would  be  more  spatial  information  over  which  to  average  from  the  longer  fins  than  the 
shorter  ones. 

The  magnitude  of  the  digitized  version  of  this  illusion  is  small.  It  is  also  very  slight  in  the  MTF  filtered  object  (Figure 
50,c,3l  but  stronger  in  the  low-pass  filter  results  (Figure  50.C.4.5I.  These  distortions  can  be  seen  more  clearly  in  the 
contour  plot.  Figure  51.  The  shorter  line  length  with  the  longer  fins  is  not  seen  in  the  blurred  image  shown  in  Appendix 
C.  .Note  also  that  the  blurred  image  still  appears  to  provide  the  illusion. 

12.3.2  The  Ponzo  Illusion 

The  original  Ponzo  illusion  (Ponzo,  1912)  contains  two  parallel  lines  equal  in  length  which  are  seen  as  unequal  in  length 
when  they  are  placed  between  two  converging  lines  ( Figure  52, a). 

The  results  of  filtering  this  illusion  using  the  MTF  and  low-pass  filter  are  shown  in  Figure  52,c.d.e.  The  up[>er 
horizontal  line  (apical  I  is  extended  into  the  vertical  oblique  lines,  an  effect  not  seen  w  ith  the  lower  horizontal  line 
(distal  I.  Note  the  decreased  intensity  at  the  ends  of  both  lines.  Thus,  the  apical  line  seems  longer  from  such  spatial 
extension;  the  distal  line  appears  shorter.  This  particular  result  is  consistent  with  the  experimental  findings  of  Quina 
and  Pollack  ( 1972),  who  found,  in  addition  to  overestimations  of  the  length  of  the  apical  line,  that  the  length  of  the 
distal  line  was  consistently  underestimated. 

Attneave  1 19721  pointed  out  that  the  distance  between  the  horizontal  and  oblique  lines  was  an  important  variable 
relating  to  the  magnitude  of  the  illusion.  Variations  of  the  Ponzo  illusion  similarly  filtered  show  such  changes. 

However,  the  general  distortions  persist  with  low-pass  filtering.  Shultz  ( 19761  noted  that,  in  addition  to  the  standard 
distortions,  the  distance  increased  between  the  two  horizontal  lines.  The  intensity  distributions  of  the  two  horizontal 
lines  of  the  filtered  Ponzo  illusion  of  Figure  52,e,d.e  show  that  effect.  The  increased  distance  between  the  average 
intensity  values  of  the  two  parallel  horizontal  lines  of  Figure  49  is  also  evident. 

1 2.3.3  Variations  of  the  Horizontal-Vertical  (H-V)  Line  Illusion  With  and  W  ithout  a  Frame 

Perhaps  the  simplest  geometric  illusion  is  a  vertical  line  which  appears  longer  when  it  intersects  a  horizontal  line  of 
equal  length  to  form  a  +  .  T,  or  L  (Kick.  1851 1. 


The  results  of  filtering  the  T  configuration  of  the  H-V  illusion  were  shown  in  Section  9.2.  The  vertical  line  is  about  4 
percent  longer  than  the  horizontal  line  after  MTF  filtering,  measured  from  the  farthest  distance  of  the  peaks  of  the 
contours  of  Figure  20,b.  However,  the  filtered  image  reveals  about  an  1 1  percent  increase  in  the  length  of  the  vertical 
line  for  the  farthest  distance  of  the  contours  of  the  4  by  4  low-pass  filtered  plot  I  Figure  2 1  ,d  I.  Suto  1 1959),  reported  by 
Oyama  1 1960),  had  subjects  compare  the  length  of  each  line  of  the  inverted  T  with  a  standard  horizontal  line.  The 
horizontal  line  length  was  underestimated  by  about  4  percent  and  the  vertical  line  length  was  overestimated  by  1 4 
percent.  This  close  agreement  between  the  filtered  image  and  experimental  findings  suggests  again  that  the  distortions 
may  be  due  to  low-spatial  frequency  channels  which  provide  the  spatial  information  used  in  the  estimation  of  perceived 
line  length. 

The  T  configuration  of  the  illusion  changes  with  orientation.  The  vertical  line  of  a  T  rotated  90  degrees  is  seen  as 
shorter  I  kunnapas.  1955,b).  This  effect  is  shown  in  Figures  53  and  54  where  the  intensities  and  the  length  of  the 
vertical  lines  are  greater  than  those  of  the  horizontal  lines. 

Several  researchers,  e.g.  Kunnapas  1 1955, a  I  and  Wenderoth  and  Beh  1 1977),  have  investigated  the  effect  of  putting 
frames  around  variations  of  the  L  or  T  line  figures.  The  general  effect  of  rotating  such  figures  in  frames  is  to  change  the 
apparent  relative  length  of  certain  components.  These  changes  in  length  depend  upon  the  size  and  shape,  etc.,  of  the 
frame.  Slight  variations  in  the  intensity  distributions  of  the  lines  in  the  L  figure  with  and  without  surrounding  lines  are 
show  n  in  Figure  55.  Note  also  the  difference  in  intensity  distributions  of  the  same  filtered  images  of  the  texture  patterns 
shown  in  Section  1 1 .3.2  due  to  their  proximity  to  other  objects  (e.g..  Figure  39, b(.  These  results  suggest  that  some 
interactions  between  figures  and  frames  may  be  accounted  for  by  the  spatial  interactions  that  occur  due  to  the  spatial 
filtering  properties  of  the  visual  system. 

12.3.4  Simple  Line  Illusions 

Even  the  simplest  line  patterns  can  produce  distortions.  The  two  examples  shown  in  Figure  56, a. b  are  from 
Miiller-Lyer  ( 1889)  and  Wundt  ( 18981.  In  Figure  56, a  the  centre  segment  of  the  line  on  the  right  appears  smaller  than 
the  centre  segment  on  the  left.  The  opposite  distortion  occurs  in  Figure  56, b.  The  line  on  the  right  appears  smaller  than 
the  centre  section  of  the  line  on  the  left. 

The  perceived  distortions  are  seen  in  the  numerical  printouts  (Figure  56,c,dl  and  the  contour  plots  (Figure  5().e.f  I  of  the 
MTF  filtered  illusions.  (To  compare  the  size  of  similar  contours,  measure  from  the  outside  of  the  image,  the  lowest 
level,  to  the  inside  of  the  image.  Measurements  made  on  the  smallest  diamond-shaped  objects  in  the  left  of  Figure  56.e.f 
are  at  the  highest  contour  level  and  are  not  found  in  the  object  at  the  right.  Therefore,  the  next  lowest  contour  is  used 
for  the  appropriate  comparison.  I 

12.4  Illusions  of  Misjudged  Size  and  Distance 

12.4. 1  The  Titchener  Illusion 

When  two  circles  of  equal  size  are  each  surrounded  by  circles  of  different  sizes,  the  circle  surrounded  by  the  smaller 
circles  appears  larger  than  the  circle  surrounded  by  the  larger  circles.  This  illusion  is  called  the  Titchener  circles 
I  Wundt,  1898).  Variations  of  this  illusion  use  squares  (Obonai.  19541,  as  shown  in  Figure  57. a,  1  ,b.  1 .  Here  the  squares 
on  the  left  have  different  perceived  size  than  the  squares  on  the  right  of  the  same  size.  The  square  surrounded  by  the 
larger  squares  ap[>ears  smaller  and  the  square  surrounded  by  the  smaller  squares  appears  larger.  These  effects  are 
shown  in  the  difference  between  the  intensity  values  in  the  numerical  printouts  and  the  size  of  the  contour  plots  after 
MTF  filtering  (Figure  57, a. 2. b, 2, a. 3, b. 31. 

12.4.2  An  Illusion  of  Misjudged  Distance 

An  illusion  reported  by  Lbbinghaus  ( 19021  is  shown  in  Figure  58. a.  Here  the  bottom  pair  of  shorter  parallel  lines 
apfiears  farther  apart  than  the  larger  upper  pair  of  parallel  lines.  The  perceived  distortion  is  evident  in  the  numerical 
computer  printout  shown  in  Figure  58, h.  The  smoothed  lines  show  average  intensity  shifts  that  reduce  the  distance 
between  the  upper  lines,  while  the  reverse  is  true  for  the  lower  pair  of  lines. 
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12..’)  Illusion  of  Nonparnllcl  Linos:  the  Zollner  Illusion 

One  illusion  that  (loos  not  appear  to  bo  explained  by  the  kind  of  spatial  filtering  that  has  been  |  resented  here  is  the 
Ziillner  illusion  tZbllner  1862).  As  shown  in  the  digitized  version  of  the  Zollner  illusion  (Figure  7>6,a|.  the  three  sections 
formed  by  the  parallel  lines  appear  to  be  bent  in  opposite  directions.  This  illusion  is  much  stronger  in  nondigitized 
form.  The  MTF  and  low-pass  filtered  pictures  shown  in  Figure  liO.b.c  do  not  show  any  such  distortion.  If  anything,  the 
intensity  distributions  go  in  the  direction  opposite  to  the  perceived  distortion. 

W  In  does  spatial  filtering  not  appear  to  work  with  this  illusion?  This  illusion  and  its  variations  have  one  major 
difference  from  any  other  illusion  which  involves  lines  that  intersect  with  angles.  The  short  lines  that  intersect  the  long 
parallel  lines  do  so  at  the  same  angle  with  the  same  lines.  This  is  not  true  of  other  geometric  illusions,  such  as  the 
laikicsh  1 1  ‘>22 1  or  Orbison  ( 1 636),  having  many  intersecting  lines.  Therefore,  the  uniform  effects  of  spatial  filtering 
occur  on  all  the  same  components  of  the  Ziillner  illusion  but  act  differentially  on  the  other  geometric  illusions. 

However,  spatial  filtering  affects  the  strength  of  the  illusion  upon  rotation. 

The  variation  of  the  Ziillner  illusion  may  be  too  coarsely  quantized  for  this  analysis,  or  the  filtered  image  causes  and/or 
allows  another  mechanism  to  create  the  perceived  distortion.  Further  research  is  needed.  Clearly  ,  in  comparison  to  all 
the  other  results  on  the  filtering  of  illusions,  the  Ziillner  illusion  and  its  variations  scent  to  be  in  a  class  by  themselves. 

12. f)  Illusions  of  Contrast 

12. <i.  I  The  Hermann  Grid  Illusion 

I  he  Hermann  Grid  illusion  (Hermann.  18701  is  shown  in  Figure  60, a.  Note  the  grey  patches  (reversed  contrast)  that 
appear  in  the  white  intersections.  These  grey  patches,  almost  completely  absent  in  the  fovea,  are  strongest  w  ith 
peripheral  viewing.  This  observation  suggests  that  low-pass  filtering  in  the  peripheral  retina  contributes  to  the  illusion. 

A  digitized  version  of  the  Hermann  Grid  illusion  is  shown  in  Figure  60. b.  The  results  of  spatially  filtering  this  illusion 
are  shown  in  Figure  60, c  where  real  intensities  at  the  intersections  are  clearly  seen.  Figure  23. d,  1  shows  another 
variation  of  this  illusion.  Note  the  grey  patches  at  each  intersection  of  the  white  squares  in  the  MTF  filtered  picture 
I  Figure  2.‘Ld.3l.  They  grey  patches  correspond  to  the  perceived  illusory  patches.  (The  reader  should  note  that  this 
illusion  is  also  present  at  the  intersections  of  the  figures  of  Figure  23. 1  A  space-variant  filter  would  be  needed  to  produce 
the  gradations  of  brightness  that  are  seen  by  the  spatially  inhomogeneous  retina. 

Kotuting  f  .gure  00, a  by  47>  degrees  causes  prominant  lines  of  reversed  contrast  rather  than  patches.  This  observation 
suggests  that  postretiua!  anistropi e  spatial  filtering  may  also  be  a  component  of  this  illusion. 

12. (>.2  The  Fhrenstein  Illusion 

The  F.hrenstein  illusion  I  Khrenstein.  1627)1.  shown  in  Figure  01  .a.  produces  increased  contrast  at  intersections  of  the 
lines.  Kecent  studies  by  Spilimann.  Fuld,  and  Gerrits  ( 16761  suggested  that  this  illusion  is  also  present  without  eye 
movements.  It  persists  with  dichoptic  presentation,  scotopic  vision,  and  with  low  contrast.  It  differs  from  the  Hermann 
Grid  illusion  because  it  is  seen  in  both  the  fovea  and  periphery.  These  findings  led  Spilimann,  et  al..  to  conclude  that  the 
enhanced  contrast  may  result  front  postretinal  factors. 

Spatially  filtering  this  illusion  I  Figure  Ol.bl  results  in  increased  contrast  at  the  intersections  of  the  lines  shown  in  Figure 
61  .e.  The  magnitude  of  the  illusion  decreases  if  it  is  rotated  by  47>  degrees,  which  suggests  that  anisotropic  spatial 
filtering  is  one  filtering  component  that  is  also  present. 

12.6.3  Ihe  llenussi  King  Illusion 

The  llenussi  King  illusion  (Benussi.  16021  shown  in  Figure  62. a,  I .  is  an  example  of  simultaneous  contrast.  The  part  of 
the  grey  ring  on  the  white  surround  is  seen  as  somewhat  darker  than  the  part  of  the  ring  on  the  black  surround.  The 
contrast  of  this  illusion  is  enhanced  if  the  edge  that  bisects  the  black  and  white  area  is  fixated.  This  forces  the  illusion  to 
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be  observed  in  the  periphery  which  emphasizes  low  spatial  frequencies.  Also,  placing  a  black  or  white  line  over  the 
same  edge  increases  the  contrast  of  the  illusion. 

The  ability  of  the  MTF  and  low-pass  filter  to  produce  images  corresponding  to  the  perceived  contrast  differences  is 
shown  in  Figure  62, a. 2, a. 3.  As  with  the  geometric  illusions,  the  slight  contrast  reversal  that  is  seen  from  filtering  by  the 
MTF  is  much  greater  with  the  low-pass  filter.  Numerical  printouts  of  the  low-pass  filtered  image  I  Figure  62,e,f  I 
confirm  the  perceived  contrast  differences  of  the  grey  ring. 

A  second  study  on  the  effect  of  adding  a  line  across  the  edge  that  bisects  the  black  and  white  area  was  made  ( Figure 
62, b.  I  ,c,  1  ,d.  1 ).  The  contrast  between  the  ring  and  white  surround  was  changed  so  that  there  should  be  a  difference 
between  the  contrast  of  the  white  line  and  the  white  surround. 

A  comparison  between  filtered  images  of  the  Benussi  ring  (Figure  62,b.l  1  and  an  added  white  (Figure  62, c,  1 1  and  black 
line  ( Figure  62. d,  1 1  reveals  only  slight  increased  contrast  of  the  two  sides  of  the  ring  ( Figure  62,b.c.d,2 1.  There  must  be 
at  least  one  other  mechanism  responsible  for  the  perceived  increased  contrast  with  the  added  lines.  For  example,  the 
added  lines  could  physically  separate  the  illusion  into  two  parts,  each  of  which  is  filtered  separately.  In  Section  13.3. 1 
segregation  of  parts  of  an  object  due  to  a  line  are  discussed  and  a  possible  reason  is  suggested  for  how  the  addition  of  a 
simple  line  increases  this  illusion. 

12.7  Subjective  (.’ontours:  the  Kanizsa  Triangle 

The  Kanizsa  triangle  (the  Kanizsa,  1655,  version  of  the  Schumann.  1604.  illusion!  represents  a  class  of  illusions  which 
provide  contours  that  are  perceived  but  are  not  physically  present  in  the  original  object.  (See  Coren,  1672.  for  review.  I 
In  Figure  63. 1  .a,  a  black  triangle  formed  by  the  white  features  may  be  seen.  Ginsburg  ( 16751  showed  that  there  is  a 
physical  intensity  distribution  that  generally  corresponds  to  the  shape  of  the  perceived  black  triangle  after  ideal 
low-pass  filtering  (Figure  63, 1. fl.  The  intensity  distribution  of  the  black  triangle  is  enhanced  if  the  low-pass  filter  is 
prefiltered  by  the  MTF  ( Figure  63, 1  .el.  If  the  visual  system  has  similar  filtering  characteristics,  then  a  physical 
triangular  form  will  exist  in  the  filtered  image  of  some  range  of  spatial  frequencies. 

This  illusion  perists,  although  greatly  diminished,  if  the  sectored  disks  are  replaced  with  dots  (Gregory.  1672,  Figure 
63.2.al.  Similar  reductions  in  the  shape  of  the  intensity  distribution  that  form  the  illusory  triangle  are  seen  in  the 
low-pass  filtered  object  (Figure  63,2,e.fl. 

12.7. 1  Comparison  Between  Solid  and  Outline  Kanizsa  Triangle 

The  results  of  this  filtering  predict  that  it  is  the  low  spatial  frequencies  rather  than  the  high  spatial  frequencies  that 
provide  the  illusory  triangle.  This  prediction  was  tested  using  a  solid  and  outline  version  of  the  Kanizsa  triangle.  The 
original  triangles  ( Figure  64, a,  1  ,b,  1 1  are  Fourier  transformed  to  create  the  magnitude  spectra  shown  in  the  contour 
plots  of  Figure  64,a,2.b,2.  Although  the  magnitude  of  the  low  spatial  frequencies  is  greatly  reduced  in  the  outline 
triangle,  the  highest  magnitude  is  at  the  third  vertical  harmonic.  It  is  about  28  percent  of  the  same  value  in  the  solid 
triangle  (see  arrows  in  Figure  64,a,2,b,2).  The  16-level  contour  plots  after  low-pass  filtering  are  shown  in  Figure 
M,a.3,b,3.  Note  that  the  contour  plot  of  the  solid  triangle  creates  a  more  homogeneous  and  better  defined  triangular 
region  than  that  of  the  outline  triangle.  Figure  64, a, 4, b, 4  shows  filled-in  contour  values  outside  the  original  features 
that  range  from  18  to  256.  Figure  64, a, 5, b, 5  shows  a  32  by  32  pel,  16-  level  numerical  printout  of  the  window  section  of 
Figure  64, a, 4, b. 4. 

Both  the  reduced  magnitude  spectra  and  the  less  well  defined  illusory  triangle  predict  a  decreased  illusion  from  the 
outline  triangle.  This  prediction  is  confirmed  by  noting  that  the  solid  triangle  produces  a  stronger  illusory  triangle  than 
does  the  outline  triangle  at  any  viewing  distance. 

The  dependence  of  the  strength  of  the  illusion  on  the  low  spatial  frequencies  has  also  been  shown  by  Dumais  and 
Bradley  ( 16761.  Using  only  the  sectored  disks  of  the  Kanizsa  triangle,  they  found  that  the  strength  of  the  subjective 
triangle  increased  as  the  size  of  the  object  decreased  from  18.6  to  1 .2  degrees  of  visual  angle.  The  strongest  subjective 
triangles  were  reported  for  a  pattern  of  small  size  ( 1 .2  to  4.8  degrees)  under  very  dim  illumination  (0. 1(1  log  lux  I 
conditions  that  make  the  filtering  characteristics  of  the  visual  system  low-pass  rather  than  band-pass  (see  Section  5.2. 1 1. 


It  is  not  clear  whether  only  one  band  of  low  spatial  frequencies  provides  the  illusion  in  this  and  other  subjective 
contours.  There  may  be  other  mechanisms,  i.e.,  other  bands  of  spatial  frequencies  that  also  contribute  to  the  sharpness 
of  the  perceived  illusion,  as  discussed  in  Section  12.8.3.  Further  research  using  spatial  filters  made  from  experimental 
data  about  the  channel  filters  is  required. 

12.8  Discussion 

The  demonstrations  with  the  MTF  and  band-pass  spatial  filters  have  suggested  that,  with  the  exception  of  the 
Zollner-type  illusion,  the  geometric  distortions  that  are  perceived  may  be  due  to  two-dimensional  spatial  filtering  by 
mechanisms  in  the  visual  system. 

In  general,  all  the  MTF  filtered  images  of  geometric  illusions  so  far  show  some  spatial  distortions  that  are  consistent 
with  perceived  distortions.  The  distortions  of  the  filtered  images  increase  with  further  spatial  filtering.  As  with  the 
previous  filtering  of  form  and  texture,  the  particular  shape  and  bandwidth  of  the  filter  were  not  critical.  However,  there 
are  small  changes  in  the  intensity  distributions  of  the  filtered  images  whose  importance  have  to  be  determined  from 
further  research.  These  small  differences  were  found  between  the  MTF(LI  and  MTFlH  I  spatial  filters,  e.g..  Figure  53. 
Therefore,  since  the  only  difference  between  these  filters  is  the  low  frequency  attentuation  characteristic,  the  amount  of 
lateral  inhibition  and  the  amount  of  contrast  that  contributes  to  the  degree  of  distortion  in  these  objects  are  clearly 
important. 

In  addition  to  the  general  filter  shape,  the  size  of  the  illusion  also  has  some  effect  on  the  magnitude  of  the  distortion. 

For  example,  the  MTF  spatial  filters  used  were  scaled  to  show  distortions  at  and  just  above  threshold  with  objects 
about  0.5  degrees  in  size.  Increasing  the  size  of  the  objects  to  2  degrees,  and  using  the  same  shape  MTFlH  I  filter,  shows 
predominant  effects  of  lateral  inhibition.  An  example  is  shown  in  Figure  65,  a  with  a  large  size  Ponzo  illusion.  Here  the 
MTF  causes  only  slight  smoothing  (Figure  65, b I,  with  low  intensity  light  and  dark  regions  surrounding  the  lines  that 
can  be  seen  after  log  transformation  (Figure  65,cl.  Examination  of  the  MTFlH IL  picture  reveals  that  the  upper 
horizontal  line  is  slightly  shorter  by  1  pel  when  compared  with  the  lower.  This  is  opposite  to  the  perceived  distortion. 
But,  this  object  was  filtered  using  a  spatially  homogeneous  filter.  The  retina  is  not  perfectly  spatially  homogeneous  over 
2  degrees,  and  this  filtered  object  does  not  represent  the  spatial  information  available  to  other  bandpass  mechanisms  of 
the  visual  system.  It  merely  demonstrates  the  difference  that  one  can  expect  to  find  in  different  size  objects  filtered  with 
similar  spatial  filters.  However,  when  this  object  is  low-pass  filtered  ( Figure  65,d,el.  the  perceived  distortion  is  evident 
as  before. 

The  distorting  effects  of  patterns  similar  to  the  illusions  have  also  been  demonstrated.  Furthermore,  it  should  be  clear 
from  the  filtered  dot  patterns  shown  in  Figures  23,b-g  and  26  why  dots  or  other  fragmented  objects  arranged  in  the 
shape  of  these  geometric  illusions  provide  the  perceived  distortions  (if  the  illusion  distortions  are  judged  from  the  low 
spatial  frequencies  of  the  objectsl. 

In  general,  these  demonstrations  have  shown  the  relative  importance  of  filter  shape,  bandwidth,  size  and  configuration 
of  the  illusion,  and  contrast.  These  are  all  interacting  variables,  some  more  important  than  others.  It  is  clear  that 
researchers  will  have  to  be  very  careful  to  control  these  variables  before  general  statements  can  be  made  about  visual 
distortions.  The  techniques  presented  here  may  offer  firm  quantitative  predictions  if  these  variables  are  controlled.  All 
of  these  techniques  can  be  refined  and  research  is  needed  to  determine  which  information  in  the  filtered  image  the 
observer  uses  for  length  of  line,  for  example,  that  can  be  related  to  the  numerical  computer  printouts  and  contour  plots. 
An  initial  attempt  to  determine  that  information  is  discussed  in  Section  12.8. 1 . 

Does  the  concept  of  spatial  filtering  represent  a  unitary  theory  for  perceived  distortions?  The  answer  is  both  yes  anti  no. 
Since  the  visual  system  has  to  process  spatial  information  with  band-limited  spatial  mechanisms,  some  distortions  like 
those  demonstrated  here  must  be  present.  In  that  sense,  filtering  is  a  unitary  theory  which  differs  from  the  other  spatial 
theories  that  have  been  presented. 

Certain  distortions  have  been  shown  that  cannot  be  accounted  for  by  blurring  or  smoothing  theories.  The  optics  of  the 
visual  system,  as  discussed  in  Section  5.2.1,  are  good  enough  to  pass  the  detail  of  objects  that  are  processed  at  higher 
levels.  Theories  of  lateral  inhibition  are  inadequate  to  account  for  all  of  the  |>erreived  distortions.  Since  the  known 


mechanisms  in  the  visual  system  are  of  finite  size,  there  also  has  to  be  some  spatial  summation.  As  Ginsburg  ( 19761 
pointed  out.  the  spatial  filters  that  have  been  used  incorporate  both  lateral  inhibition  and  high-frequency  attenuation 
(spatial  summation).  In  sum.  the  proponents  of  various  spatial  theories  of  geometric  distortions  have  been  only 
partially  correct. 

The  death-knell  of  a  cognitive  theory,  i.e.,  inappropriate  scaling  (Gregory,  1963)  as  a  unitary  theory,  has  been  sounded 
for  many  years  and  the  numerous  counter-examples  will  not  be  presented  here.  The  interested  reader  is  referred  to 
Robinson  ( 19721. 

It  is  interesting  to  note  that  Gregory  and  Harris  ( 19751  presented  the  results  of  an  experiment  that  was  supposed  to 
show  that  the  spatial  filter  theory  failed  when  the  inappropriate  scaling  theory  predicted  a  negligible  illusion  from  a 
Miiller-Lyer  figure  under  certain  scaling  conditions.  However,  Ginsburg  (19761  pointed  out  that  Gregory  and  Harris 
created  the  conditions  under  appropriate  scaling  whereby  the  fins  of  the  Miiller-Lyer  figures  appear  at  “right  angles"  in 
depth  planes  away  from  the  shafts.  Under  reversed  stereo  projection ,  depth  was  “labile."  julesz  (19711  demonstrated 
the  destruction  of  the  Miiller-Lyer  illusion  by  separation  of  the  shafts  into  separate  depth  planes  by  means  of  random 
dot  stereograms.  Therefore,  Gregory  and  Harris's  results  could  have  been  predicted  from  the  two-dimensional  filter 
theory.  If  the  illusion  features  could  not  interact  spatially  in  the  same  depth  plane,  then  the  filtering  would  act  on  the 
pattern  elements  in  each  depth  plane.  However,  they  would  not  be  able  to  distort  each  other's  spatial  distribution  and 
would  not  produce  illusion  distortions. 

Since  spatial  filtering  of  the  kind  presented  here  cannot  offer  an  explanation  of  the  Zollner-type  illusions,  it  cannot  be 
considered  a  unitary  theory.  Nor  would  the  author  expect  it  to  be.  Clearly,  factors  such  as  pattern  of  eye  scan,  individual 
strategies  to  determine  the  spatial  position  and  extent  of  lines,  and  even  inappropriate  scaling  effects  (if  based  upon  the 
filtered  image,  e.g.,  different  intensities  and  sizes  corresponding  to  different  depth  planes,  as  was  discussed  in  Section 
1 1.6)  would  seem  to  have  some  input  to  the  final  response  of  the  observer  (Ginsburg.  1971,a,b|.  However,  the  filtering 
results  presented  here  suggest  that  these  may  be,  at  best,  second  and  third  order  variables. 

The  techniques  presented  in  this  dissertation  can  be  used  to  determine  the  extent  of  distortions  due  to  spatial  filtering 
and  other  variables.  For  example,  the  distortions  due  not  only  to  lateral  inhibition  can  be  determined  by  using  a 
Gaussian  low-pass  filter — a  filter  that  only  produces  spatial  summation. 

What  is  the  site  of  the  spatial  filtering  that  can  produce  such  distortions?  Clearly  any  spatial  filtering  will  cause  the  vast 
majority  of  geometric  illusions  to  have  distortions.  Forcing  judgements  of  line  length,  contrast,  etc.,  in  the  peripheral 
retina  increases  errors  due  to  the  reduced  spatial  frequency  resolution  found  in  the  periphery  .  In  general,  all  the 
distortions  change  to  some  degree  upon  orientation.  W'hen  the  distortions  were  measured  against  general  experimental 
findings,  the  degrees  of  distortion  corresponded  best  to  those  illusions  filtered  with  spatial  filters  with  bandwidth 
similar  to  those  required  to  create  generalized  forms  in  previous  chapters — two  octaves. The  only  orientation-selective 
mechanisms  in  the  visual  system  that  have  bandwidths  of  two  octaves  have  been  found  in  the  cortex  (see  Section  7.31. 
Thus,  the  cortex  may  be  the  major  site  of  the  filtered  image  from  which  distortions  are  perceived. 

12.8. 1  Predicting  the  Perceived  Length  of  the  !YIiiller-Lyer  Illusion  from  F  iltered  Images 

Pressey  and  Bross  ( 19731  reported  the  perceived  change  in  the  length  of  the  shaft  of  a  configuration  of  the  Miiller-Lyer 
illusion  having  long  and  short  fins.  There  were  nine  gaps  ranging  from  0  to  20  percent  of  the  length  of  the  shaft  between 
the  fins  and  the  shaft.  The  subjects  selected  a  line  that  appeared  to  be  equal  to  the  length  of  the  shaft  for  each  gap  size. 
The  perceived  change  in  shaft  length  plotted  as  a  function  of  gap  size  is  shown  in  Figure  ()6  by  the  inverted  U-shaped 
functions.  Cornell  ( 19781.  using  that  paradigm,  reported  an  initial  attempt  to  predict  the  perceived  length  of  a 
configuration  of  the  Miiller-Lyer  illusion  from  filtered  images.  He  used  the  same  set  of  illusions;  however,  the  subjects 
used  the  method  of  adjustment  to  match  the  length  of  a  line  to  the  perceived  length  of  the  shaft. 

The  results  of  Cornell  's  ex|>eriment  are  shown  in  Figure  Mi.  Those  experimental  curves  were  compared  to  theoretical 
curves  determined  from  filtered  images.  The  nine  illusions  were  digitally  filtered  using  one-,  two-,  and  three-octave 
ideal  and  channel  band -pass  filters  hav  ing  centre  frequencies  of  1 ,  2.  4.  8,  It),  32.  and  64  cy  cles  [s*r  size  of  the  illusion. 
The  theoretical  length  of  the  shafts  was  determined  from  the  distance  between  the  0.25.  0.50.  0.75,  and  1 .00  jiercenl 


values  of  the  peak  amplitudes  of  the  shafts  of  the  filtered  images.  The  best  fit  to  the  experimental  data,  shown  in  Figure 
66.  is  from  the  filtered  images  from  the  two-octave  channel  filters  having  centre  frequencies  of  2  and  4  cycles  |>er  size  of 
the  illusion — the  range  of  low  spatial  frequencies  that  are  predicted  from  the  previous  filtering  (see  Section  12.3. 1 1. 

Six  of  nine  illusion  configurations  were  predicted  well.  However,  the  three  configurations  having  the  greatest  gap  size 
were  not  predicted  well;  the  length  of  the  lines  of  the  filtered  images  were,  in  general,  too  short  compared  to  the 
perceived  length.  There  are  several  hypotheses  as  to  why  those  data  could  not  be  predicted.  First,  the  theoretical  length 
of  the  shaft  was  determined  from  the  filtered  images  using  filters  whose  bandwidth  was  based  on  the  overall  size  of  the 
illusion,  i.e.,  from  fin  tip  to  fin  tip.  Although  this  assumption  appears  reasonable  for  six  of  the  illusions,  the  magnitude 
of  the  perceived  illusion  may  not  be  based  on  the  overall  size  for  the  remaining  three  illusions  because  of  the  large  gaps 
between  the  fins  and  the  shaft.  It  is  possible  that  the  overall  size  of  the  illusion  with  large  gaps  is  taken  as  the  shaft  and 
one  fin.  This  hypothesis  could  be  tested  by  comparing  the  magnitude  of  the  perceived  illusion  with  and  without  one  fin. 
If  the  perceived  magnitude  of  the  illusion  remains  the  same  under  those  conditions,  then  the  theoretical  shaft  length 
should  be  determined  from  a  filtered  image  containing  only  one  fin  and  the  shaft.  This  would  reduce  the  shrinkage  of 
the  shaft  so  it  may  better  correspond  to  the  perceived  shaft  length.  Second,  the  amplitude  values  used  to  determine  the 
theoretical  shaft  length  may  not  correspond  to  those  used  by  subjects  for  those  illusion  configurations.  Third,  the 
theoretical  values  were  determined  from  filtered  images  based  on  a  single  channel  of  a  certain  shape  and  bandwidth. 
Two  or  more  channels  may  be  interacting  in  unknown  ways,  producing  the  effect  of  requiring  different  filter  shapes  and 
bandwidth  for  those  more  complex  configurations  of  the  illusion.  Of  course,  these  three  factors  may  be  interacting 
simultaneously.  Future  research  will  be  required  to  test  these  hypotheses. 

These  results  that  show  certain  agreement  between  some  of  the  theoretical  and  experimental  data  are  good,  even  without 
agreement  with  all  the  data.  The  general  shapes  of  the  curves  are  similar.  The  underestimation,  peaking  to 
overestimation,  and  then  the  decreasing  of  the  perceived  shaft  length  is  predicted  by  the  filtered  images.  The 
overestimation  and  underestimation,  shown  to  be  between  ±  15  percent  of  the  length  of  the  shaft,  is  predicted.  Finally, 
the  location  of  zero  and  peak  values  of  the  perceived  shaft  length  are  closely  predicted.  In  sum.  these  initial  results, 
attempting  to  quantitatively  relate  the  perceived  length  of  a  complex  object  from  filtered  images  based  upon  biological 
data,  are  promising. 

12.8.2  Filtered  Images.  Eye  Scans,  and  Illusions 

Another  important  clue  about  the  possible  site  where  the  distortion  is  already  present  comes  from  eye  scans  that  have 
been  measured  on  illusions.  It  seems  clear  that  the  low  spatial  resolution  of  the  peripheral  retina  forces  the  observer  to 
make  eye  scans  to  place  the  fovea  over  various  parts  of  an  object  to  provide  greater  detail. 

It  may  be  that  the  mechanism  that  guides  the  foveal  fixation  is  already  dealing  with  low  spatial  frequency  images. 
Yarbus  1 10671  showed  eye  scan  paths  for  the  Miiller-Lyer  illusion  (Figure  671.  (He  also  found  that  the  illusion  persisted 
without  eye  movements.  Thus,  theories  attributing  geometric  distortions  to  inappropriate  eye  movement  appear 
untenable.  I 

The  author  calculated  the  average  distance  between  the  different  fixation  spots  in  the  traces  of  the  eye  scans  shown  in 
Figure  67.  The  ratio  of  the  average  distance  between  the  large  fixation  spots  of  the  inward  to  outward  going  fins  is 
about  0.80.  1’ressey  ( 1072)  graphed  data  from  Heymans  ( 18061  that  showed  a  0.84  ratio  between  the  distance  of  a  line 
with  fins  at  45  degrees  outward  and  fins  at  'HI  degrees.  The  ratios  from  the  computer  printouts  are  0.85  [lMTFl  H  l]  . 

0.78  [MTFfLl], and 0.75  [MTF(H!I5  by  5)] .  These  values  were  determined  by  counting  the  number  of  |>els  of 
approximately  the  same  intensity  at  the  ends  of  the  shafts  of  the  filtered  images.  The  illusion  was  coarsely  quantized 
and  the  end  points  that  an  observer  uses  in  making  line  length  judgements  have  yet  to  be  fully  determined.  However, 
the  values  presented  here  are  close  enough  to  other  experimental  results,  e.g..  (Cornell  ( 10781,  to  suggest  that  it  is  the 
low-pass  filtered  image  that  may  guide  certain  eye  scans. 

Further  evidence  for  certain  eye  scans  being  directed  by  spatial  information  in  filtered  images  is  given.  Eye  scan  paths 
of  a  portrait  of  a  young  girl  ( Figure  68,a.bl  and  a  bust  of  Nefertiti  (Figure  hO.a.bl  are  from  Yarbus  ( 1067 1.  These 
pictures  were  digitized  ( Figures  68.c,60,c  I  and  filtered  as  before  to  create  the  filtered  images  shown  in  Figures  (>8. 

60,d,e.  Note  that  the  density  of  eye  scans  corresponds  well  to  the  regions  of  large  contrast  differences  of  the  major 
features,  e.g.,  nose,  eyes,  and  mouth,  seen  in  filtered  images. 


The  suggestion  that  certain  eye  scans  may  be  directed  by  spatial  information  contained  in  a  filtered  image  is  also 
supported  by  the  fact  that  the  fovea  is  directed  to  objects  in  the  periphery  that  have  relatively  low  resolution. 

12.8.3  Why  Are  the  Details  of  Illusions  Seen  as  Distorted? 

The  perceived  distortion  of  many  illusions  shown  in  this  dissertation  corresponded  well  to  the  distortions  found  in  their 
low-pass  filtered  images.  Using  this  approach,  Cornell  I 1978)  found  that  the  theoretical  data  from  the  low-pass  channel 
filtered  illusions  predicted  well  the  perceived  length  of  the  shaft  for  six  of  nine  configurations  of  the  Miiller-Lyer 
illusion.  However,  it  is  reasonable  to  ask  why,  if  these  data  suggest  that  the  perceptual  space  for  distortions  of  illusions 
is  the  low-pass  filtered  image,  the  details  of  the  illusions  are  seen  as  distorted,  i.e..  the  high  spatial  frequencies  of  the 
objects.  This  question  is  answered  by  assuming  that  there  are  channels  in  the  visual  system,  with  a  bandwidth  of  about 
two  octaves,  that  provide  a  hierarchy  of  filtered  images  similar  to  those  shown  for  the  portraits  in  Figure  83,  as 
discussed  in  Chapter  14.  The  filtered  images  may  combine  to  create  the  global  picture  that  is  seen.  Based  upon  that 
assumption,  there  are,  in  general,  two  possible  combinational  rules  that  would  distort  the  high  spatial  frequencies. 
Details  of  the  filtered  images  from  the  higher  spatial  frequency  channels  might  follow  the  contours  of  the  filtered  images 
from  the  lower  spatial  frequency  channels,  or  they  may  be  spatially  shifted  by  inhibition  from  the  contours  of  the 
filtered  images  from  the  lower  spatial  frequency  channels.  The  author  has  tried  to  determine  experimentally  the 
combinational  rules  of  filtered  images  from  channel  filtering  by  attempting  to  relate  the  intensity  distribution  of 
combinations  of  filtered  images  to  the  perception  of  the  tile  illusion  (Schachar,  19761.  Good  agreement  between  the 
intensity  distribution  of  the  resulting  filtered  image  and  the  brightness  distribution  of  the  perceived  illusion  was 
obtained  when  the  resulting  filtered  image  was  created  from  an  envelope  of  several  filtered  images  of  which  the  intensity 
at  each  point  in  space  was  the  maximum  value  of  each  filtered  image.  This  approach  may  be  used  to  determine  if  the 
low-pass  filtered  image  can  distort  the  high-pass  filtered  image  when  they  are  combined.  Much  further  research  will  be 
needed  in  this  area  before  the  actual  combinational  rules  for  filtered  images  from  channels  in  the  visual  system  are 
known  to  fully  explain  why  the  details  of  illusions  are  seen  as  distorted. 

12.9  Summary 

This  chapter  has  provided  numerous  demonstrations  showing  filtered  images  of  illusions  that  contained  distortions  that, 
in  general,  corresponded  well  to  perceived  distortions.  The  only  exception  was  the  Ziillner  illusion.  These  results 
suggested  that  the  primary  reason  for  seeing  most  illusions  is  that  spatially  band-limited  mechanisms  in  the  visual 
system  cause  filtering  that  distorts  the  geometric  and  brightness  distributions  of  the  original  objects.  This  explanation 
was  tested  experimentally  with  very  good  initial  results.  Finally,  a  relationship  between  filtered  images  and  eye  scans 
was  postulated.. 

The  next  chapter  further  pursues  the  filtering  concept  for  perception  on  more  complex  visual  phenomena:  niultistable 
objects. 


<»«! 


CHAPTER  13 


MULTISTABLE  OBJECTS 


13.1  Introduction 

An  example  of  a  simple  multistable  object  is  monocular  rivalry,  discussed  in  Section  6.2.4,  which  occurs  when  two 
crossed  gratings  perceptually  alternate.  There  are  more  complex  objects  that  change  shape  or  can  be  interpreted  in  two 
or  more  ways  that  are  also  classed  as  multistable  objects.  Attneave  1 1671 1  provides  a  good  review  article  for  the 
interested  reader.  Can  the  concept  of  spatial  filtering  offer  any  insight  into  the  perception  of  multistable  objects?  This 
is  examined  next. 

13.2  Multistability  Due  to  the  Different  Sixes  of  Objects  in  One  Scene 

Certain  scenes  are  multistable  because  the)'  contain  objects  of  different  sizes  in  one  picture.  An  example  of  such 
multistability  is  shown  in  Figure  7(1, a.  This  picture  is  a  portion  of  "Slave  Market  with  Apparition  of  the  Invisible  Bust 
of  Voltaire"  by  Salvador  Dali.  As  Attneave  (1671 1  pointed  out,  either  the  bust  or  the  people  predominate  at  different 
viewing  distances.  With  reference  to  the  overall  size  of  the  picture,  the  bust  is  larger,  thus  having  predominantly  lower 
spatial  frequencies  than  that  of  the  people.  A  prediction  of  this  picture  in  terms  of  filtering  suggests  that  the  bust  and 
[teople  will  segregate  into  two  predominantly  different  spatial  frequency  channels.  This  notion  is  demonstrated  next. 

The  original  picture  was  sampled  into  128  by  128  discrete  picture  elements  (pels)  that  resulted  in  Figure  7().b.  The 
details  of  the  people  are  considerably  reduced  by  this  form  of  low-pass  filtering.  Additional  low-pass  (24  by  24.  Figure 
7(1. cl  spatial  filtering  eliminates  the  people,  but  the  bust  remains  well  resolved.  Further  spatial  filtering  removes  even 
the  bust  and  leaves  only  general  low  frequency  forms  from  the  original  picture  (Figure  70. dl.  (  This  intensity 
distribution  is  interesting  in  itself  in  that  it  seems  to  show  the  areas  of  the  original  picture  to  which  one  first  attends 
U{ton  initially  viewing  the  picture.  This  observation  has  been  made  by  the  author  on  many  other  patterns  that  have  been 
filtered  similarly,  and  further  suggests  that  filtered  images  may  provide  information  for  eye  scan  mechanisms  (see 
Section  1 2.8.21.  (Thus,  the  people  lie  in  a  spatial  frequency  region  greater  than  24  c/pw.  whereas  the  bust  can  be  found 
from  about  8  to  24  c/pw.  These  results  are  further  demonstrated. 

l’he  bust  was  enlarged  to  twice  the  size  of  the  original  picture  in  order  to  capture  more  of  the  detail  of  the  faces  found  in 
the  eye  sockets  of  the  bust  ( Figure  70, el.  As  found  in  the  previous  study,  the  faces  are  lost  with  moderate  filtering  and 
the  bust  remains  with  further  low-pass  filtering. 

hich  form  will  predominate  at  different  viewing  distances  can  perhaps  be  best  explained  in  terms  of  the  filtering 
characteristics  of  the  visual  system:  contrast  sensitivity  function  and  retinal  inhomogeneity.  Contrast  sensitivity  is 
generally  greatest  at  about  1  to  5  c/ d  of  visual  angle.  Therefore,  as  the  viewing  distance  changes,  the  predominant 
spatial  frequency  band  for  each  of  the  forms  will  be  seen  at  different  levels  of  visual  sensitivity.  View  Figure  7(1. e  so  that 
the  faces  are  within  the  fovea. about  1  degree.  The  ratio  between  the  sizes  of  the  faces  and  the  bust  is  about  1 :8. 
Therefore,  the  third  harmonic  of  each  face  (determined  as  approximately  the  minimum  spatial  frequency  necessary  for 
the  recognition  of  faces  from  previous  studies,  see  Section  10.21  is  at  about  3  c/d,  whereas  the  third  harmonic  of  the 
bust  is  at  about  0.38  c/ d.  Since  the  major  portion  of  the  bust  falls  outside  the  fovea  where  retinal  inhoniogeneitv  reduces 
contrast  sensitivity  to  high  spatial  frequencies  even  further,  it  seems  reasonable  that  the  faces  dominate  our 
|>erception  at  such  close  viewing  distances  (when  viewed  without  eye  scansl. 

If  the  picture  is  viewer!  at  a  distance  so  that  the  bust  subtends  about  2  degrees,  the  third  harmonic  of  the  bust  is  3  c/d. 
whereas  the  third  harmonic  of  the  faces  is  24  c/d.  Since  contrast  sensitivity  is  reduced  greatly  at  24  c/d  compart'd  to 
3  c/d.  the  bust  seems  to  predominate,  although  the  faces  are  still  recognizable.  At  a  third  viewing  distance  so  that  the 
bust  subtends  1  degree  of  visual  angle,  the  third  harmonic  of  the  bust  is  6  c/d.  However,  the  third  harmonic  for  the 
faces  is  ‘18  c/d,  near  the  limit  of  our  normal  visual  acuity.  At  this  distance,  the  bust  is  clearly  dominant  over  the  faces. 

I  here  are.  of  course,  other  factors  such  as  set.  selective  attention,  and  other  picture  clues  that  can  contribute  to  whether 
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the  (teople  or  the  bust  is  more  noticeable.  Note  that  the  figure  of  the  ntan  on  the  left  side  of  the  picture  resting  his  head 
on  his  hand  is  about  the  same  size  as  that  of  the  bust  and  would  cause  a  similar  spatial  frequency  band  to  be  attended  to 
if  noticed  first  land  vice  versa).  A  similar  situation  exists  for  the  |>eople  figures  in  the  background.  It  seems  that  Dali 
knew  quite  well  how  to  induce  the  v  isual  system  to  perceive  different  size  objects  to  create  a  multistable  picture 
(Fisher,  l%7l. 

The  adequacy  of  the  low  spatial  frequencies  for  the  classification  of  forms  can  also  be  demonstrated  using  these 
patterns.  If  the  large  busts  are  viewed  at  a  distance,  they  all  look  alike  lat  about  4.'>  yardsl.  Moving  slowly  towards  the 
pictures  causes  the  busts  on  the  right  to  be  seen  as  different  f  »m  the  ones  on  the  left  as  their  higher  spatial  frequencies 
rise  above  their  threshold.  The  busts  are  recognized  at  about  1.1  yards,  when  the  third  harmonic  is  about  l<>  c/d  and  the 
fourth  harmonics  are  at  about  42  c/d  and  Mi  c/d.  respectively.  These  results  are  consistent  with  other  findings  that  have 
shown  that  about  four  harmonics  are  sufficient  for  form  recognition  iGinsburg.  I ‘>71  ,a.b;  Woodhouse  and  Campbell. 
l«)77l. 

Note  that  the  recognition  of  the  busts  requires  a  higher  spatial  frequency  than  that  of  the  faces.  The  studies  of  filtered 
|>ortraits  (Chapter  ltd  suggest  that  the  critical  features  of  the  face  for  recognition  are  the  |>osition  of  the  eyes.  nose,  and 
mouth.  Since  the  bust  is  a  three-quarter  profile  and  there  is  only  a  small  separation  between  the  nose  and  the  mouth, 
these  features  would  Ite  at  a  higher  spatial  frequency  than  those  of  a  frontal  view  of  the  other  faces  in  the  picture. 

1.1/1  The  Bisected  Circle 

<  >ne  of  the  most  fundamental  patterns  which  exhibits  figure-ground  properties  is  a  circle  bisected  by  a  ser|M*ntine  line 
as  shown  in  Figure  71  .a.  Kither  side  can  become  figure  with  equal  ease.  Note  that  the  side  that  becomes  the  figure 
ap|M‘ars  to  he  brighter  than  the  other  side  and  that  the  serpentine  line  appears  to  belong  to  the  figure,  not  to  the  ground. 
Informal  observation  of  this  pattern  suggests  that  eye  scans  may  not  play  a  very  large  role  in  determining  which  side  is 
seen  as  figure  and  which  is  seen  as  ground.  It  may  be  that  some  unbalanced  information  front  the  object  is  needed  in 
order  for  some  mechanism  to  segregate  one  side  from  the  other. 

I"  igure  .  I  .b.c.d  shows  simpler  versions  of  this  object  made  to  further  investigate  this  kind  of  figure-ground  separation 
when  the  serpentine  line  is  replaced  by  a  straight  line.  The  asy  mmetric  object  shown  in  Figure  7 1  ,b.c  results  in 
figure-ground  more  readily  than  the  symmetric  object  shown  in  Figure  72. d.  This  demonstration  suggests  that  the  basic 
information  being  used  for  the  separation  of  figure  and  ground  in  these  objects  is  size  or  area  differences  between 
sections  of  the  objects  created  by  the  |»osition  of  the  line.  The  filter  concept  suggests  that  there  is  some  band  of  spatial 
frequencies  that  form  an  image  which  reflects  the  differences  between  size  or  area  in  these  patterns.  To  examine  this 
notion,  these  digit ized  versions  ( Y  igure  72. a.  I -d.  1 1  were  sequentially  low -pass  filtered  and  the  images  examined  for 
global  information  that  could  be  used  to  segregate  sections  of  the  patterns.  Such  information  was  found  in  the  lowest 
spatial  frequencies  of  the  patterns,  in  particular,  the  fundamental  frequency  as  seen  in  the  plots  of  the  filtered  images  of 
f  igure  72.a.2-d.2.  Note  how  this  very  low  spatial  frequency  information  reflects  the  asy  mmetry  of  these  patterns. 

I  hese  results  suggest  that  the  distribution  of  spectral  energy  in  a  filtered  image  formed  by  the  very  low  spatial 
frequencies  can  prov  ide  spatial  regions  for  the  segregation  of  figure  from  ground. 

I  hese  results  are  interesting  la-cause  they  provide  a  function  for  a  region  of  spatial  frequencies  that  were  not  needed  for 
the  demonstrations  in  Chapters  ')  and  10.  In  those  studies,  the  removal  of  the  very  low  spatial  frequencies  from  the 
object  still  resulted  in  recognizable  forms,  although  the  contrast  was  reduced.  These  images  found  from  the  verv  low 
spatial  frequencies  (less  than  I  cycle  [>er  object  I  could  also  provide  the  position  (phase!  information  with  which  the 
lines,  i.e..  high  spatial  frequencies,  of  each  section  of  the  patterns  could  become  associated  (sis-  Sis- lion  12. b.. 'll. 

F urthermore.  suppressing  the  intensity  distribution  in  one  area  by  inhibition  or  selective  attention  would  "highlight'' 
the  other  area  causing  it  to  become  the  figure.  Thus,  it  may  be  that  the  very  low  spatial  frequencies  "sfxitlight"  (In¬ 
different  areas  of  objects  with  which  the  images  from  the  higher  spatial  frequencies  can  be  associali-d:  one  pattern 
becomes  the  figure  and  the  other  the  ground.  This  post  his-  explanation  suggests  w  hy  the  figure  seems  brighter  than  the 
ground  and  also  on  which  side  a  common  line  or  contour  should  appear. 

I.1..1. 1  An  Kxperiment:  Segregation  of  Two  Areas  by  a  l.ine 


I  he  segregation  property  of  lines  in  objects,  e.g.,  the  creation  of  local  areas  or  regions  for  analv  sis.  discussed  in 


Section  13.3.  has  a  testable  prediction:  Certain  information  contained  in  an  area  of  an  object  should  be  harder  to 
extract  if  the  object  is  physically  partitioned  by  a  sharp  dividing  line  into  two  different  areas.  For  example,  the  light  and 
dark  bars  of  a  square-wave  grating  are  easily  seen  having  only  one  db  difference  in  contrast.  Does  such  a  small 
difference  produce  similar  results  if  a  line  is  put  between  the  bars  putatively  segregating  them?  This  was  determined  in 
the  following  experiment. 

A  2-degree  bipartite  field  with  and  without  a  6-minute  line  placed  over  the  common  edge  of  the  bipartite  field  was 
presented  on  an  oscilloscope  to  two  subjects.  The  task  of  the  subject  was  to  adjust  the  contrast  of  one  side  of  the  field, 
starting  at  zero  contrast,  until  the  difference  in  contrast  between  the  two  sides  was  just  detectable.  The  procedure  w  as 
then  re  terser! .  starting  with  a  high  contrast  bipartite  field.  The  contrast  was  decreased  until  no  contrast  difference  was 
seen  between  the  two  sides.  The  measurements  were  alternated  for  30  trials.  Voltage  readings  that  corresponded  to  the 
contrast  were  recorded  for  each  measurement.  The  average  voltage  measurement  without  the  line  was  84.0  mv  (SI)  = 

1 1 .001  for  the  first  subject,  and  80.7  mv  (SD  =  14.41  for  the  second  subject.  The  average  voltage  with  the  line  was  300 
mv  (SI)  =  00. 41  for  the  first  subject  and  783. 1  mv  (SD  -  144.41  for  the  second  subject.  Thus,  the  first  subject  required 
about  4  times  ( 1 1.2dB)  more  contrast  to  discriminate  the  contrast  of  the  bipartite  field  with  than  without  a  line.  The 
second  subject  needed  about  10  times  ( 10.7 dBImore  contrast  to  perform  the  same  task  with  the  line.  These  results 
show  that  the  contrast  information  that  was  available  from  the  bipartite  field  without  the  line  was  greatly  reduced  w  hen 
the  line  was  present.  These  results  also  suggest  that  the  line  segregated  the  bipartite  field  into  two  separate  regions,  at 
least  in  terms  of  contrast  information.  Further  experiments  are  planned  to  determine  the  spatial  frequency  and  contrast 
properties  of  lines  that  provide  that  segregation. 

13.4  The  Kuhin  Face-Vase  Object 

The  Kuhin  Vase- Face  object  (or  Candlestick  illusionl  (Rubin.  10211  is  another  example  of  multistability  because  of 
figure-ground  reversal.  As  shown  in  Figure  73, a.  either  a  vase  or  the  profiles  of  faces  in  silhouette  can  be  seen.  As  one  is 
seen  to  predominate  as  the  figure,  the  other  is  seen  as  the  background,  or  vice  versa.  The  multistability  of  this  object 
has  been  explained  previously  in  terms  of  filtered  images  (Ginsburg.  1071  .a. b:  Ginsburg,  et  al.,  10761.  It  was  suggested 
that  if  the  observer  attended  to  the  lower  spatial  frequencies,  then  the  smoothed  image  would  provide  sufficient 
information  for  the  perception  of  the  vase  (or  candlestick!  but  not  sufficient  information  for  the  |>ereeption  of  the  faces 
( Figure  73, d I.  However,  if  the  observer  was  attending  to  the  higher  spatial  frequencies,  for  example,  by  focusing  his 
attention  on  the  details  of  the  pattern,  the  faces  were  reported. 

The  ambiguity  of  the  Rubin  face- vase  object  can  be  reduced  if  the  overall  size  of  the  vase  is  made  larger  than  the 
distance  between  the  forehead  and  chin  of  the  faces,  as  shown  in  Figure  73, e.  This  configuration  of  the  figure  appears 
to  provide  a  stronger  segregation  of  figure-ground  because  the  information  about  the  vase  and  faces  is  further  separated 
in  size,  i.e.,  spatial  frequency.  The  greater  the  difference  in  size  between  two  patterns,  the  more  easily  they  can  be 
separated  by  filtering. 

The  segregation  of  figure-ground  in  these  subjects  is  aided  by  a  difference  in  contrast  between  the  vase  and  faces.  The 
vase  informed  by  a  white  on  black  contour,  whereas  the  faces  are  formed  by  black  on  white  contours.  However, 
figure-ground  reversal  can  still  occur  w  ith  this  object  if  only  the  outline  is  present,  as  shown  in  Figure  7.3, f.  Note  that 
the  outline  still  prov  ides  low  spatial  frequencies  that  allow  the  segregation  of  the  vase  and  faces  by  filtering  lallhough 
the  low  frequencies  w  ill  be  of  lower  energy  in  the  outline  form  than  in  the  solid  form,  as  was  shown  in  Section  12.7.1 1. 

Note  also  that  the  filter  explanation  for  this  multistable  object  is  consistent  with  the  previous  filtering  results  which 
showed  that  the  low  spatial  frequencies  provide  the  basic  information  about  object  form.  Many  different  configurations 
of  this  'iject  can  also  result  in  the  perception  of  vases  or  goblets  or  candlesticks  and  silhouetted  faces.  The  lower 
spatial  frequencies  of  those  different  objects  can  provide  generalized  form  information  that  would  allow  different 
interpretations  of  the  filtered  image. 

The  reversibility  of  these  figures  can  easily  be  overcome  if  they  ure  v  iewed  at  a  distance  such  that  the  band  of  spatial 
frequencies  that  provide  the  face  information  is  attenuated  to  the  degree  that  only  the  lower  spatial  frequencies  of  the 


vuse  remain. 


13.5  Multistable  Dot  Patterns 


Atkinson,  et  al.  1 1*1731,  reported  the  perceptual  effect  of  another  type  of  muitistable  object  which  exhibited  monocular 
rivalry.  The  uniform  field  of  dots  shown  in  Figure  74, a  can  be  seen  to  shift  perceptually  from  horizontal  to  vertical 
columns,  and  sometimes  even  into  rectangular  shapes.  What  are  the  possible  bases  for  such  perceptual  shifts?  One 
|>ossibilily  is  that  different  populations  of  mechanisms  tuned  to  similar  ranges  of  spatial  frequency  and  orientation  shift 
their  relative  activity. 

The  analogue  in  transform  space  is  the  attenuation  of  certain  parts  of  the  transform.  By  shifting  the  spatial  filter  either 
vertically  or  horizontally  I  Figure  74,c,  1-f,  1 1,  some  vertical  and  horizontal  substructures  can  be  seen  in  the  filtered 
images  (Figure  74,c.2-f,2l.  Note  that  the  structure  of  the  pattern  changes  as  a  whole.  Just  as  viewing  of  two  or  three 
of  the  dots  cannot  form  a  separate  group  from  the  rest  of  the  dots,  neither  can  the  filtering.  The  “common  fate"  of  the 
[•erception  of  the  dots  also  applies  to  the  filtered  image  that  is  created  by  spatial  filtering. 

One  simple  prediction  based  on  these  results  of  filtering  is  that  adapting  one  set  of  orientation  channels  should  bias 
perception  and  cause  the  dots  to  group  in  the  other  orientation.  The  reader  can  demonstrate  this  effect  by  gazing  over 
any  one  of  the  three  gratings  shown  in  Figure  75  for  about  2  to  3  minutes  and  then  inspecting  the  original  dot  pattern, 
Figure  74,a.  The  dots  should  appear  to  be  grouped  in  a  different  orientation  than  that  of  the  adapting  grating. 

This  is  just  one  simple  example  of  orientation-selective  multistability.  The  next  example  is  more  complex. 

13. (>  Multistable  Triangles 

An  example  of  three-dimensional  multistability  is  shown  in  Figure  76. a. 2.  Initially  reported  by  Attneave  ( 1668,  16711. 
such  fields  of  equilateral  triangles  are  perceived  as  a  unit,  all  appearing  to  point  randomly  in  one  of  three  directions. 
Furthermore,  these  perceptual  fluctuations  of  orientation  are  accompanied  by  uniform  changes  in  the  shape  of  the 
triangles  (typically,  the  equilateral  triangles  are  perceived  as  isosceles  triangles!  and  the  triangles  can  appear  in  depth 
(like  a  set  of  pyramids  lit  on  one  side  by  the  sun). 

It  is  quite  surprising  that  the  triangles,  separated  by  considerable  distances,  should  all  appear  to  look  alike.  Such  v  isual 
processing  must  clearly  go  beyond  just  describing  pattern  features  and  must  involve  some  holistic  processing  over  the 
entire  visual  space  in  order  for  the  triangles  to  share  a  “common  fate."  Attneave 's  analysis  of  such  phenomena 
(Attneave.  1671.  p  7(K  "  supposelsl  the  perceptual  system  employs  something  quite  like  a  Cartesian  coordinate 
system  to  locate  and  describe  things  in  space." 

Atkinson,  et  al.  1 16731,  have  suggested  that  the  alternation  of  the  triangles  may  be  linked  to  the  same  mechanism  of 
such  visual  phenomena  as  monocular  rivalry.  Perhaps  an  investigation  that  incorporates  the  filtering  concepts 
mentioned  above  can  offer  additional  insight,  especially  as  to  a  possible  reason  why  distributed  objects  behave  in  such  a 
unitary  manner. 

The  magnitude  spectrum  of  the  triangles  is  shown  in  Figure  76,a.  I .  Note  that  the  spatial  information  of  the  triangles  is 
now  available  in  three  spatial  frequency  /orientation  regains  (channels),  containing  approximately  equal  energy. 

The  vertical  sides  of  the  triangles  are  transformed  into  the  horizontal  channel.  The  Mi-degree  and  120-degree  sides 
transform  into  the  1 2(l-degree  and  (Kl-degree  channels,  respectively.  Previous  filtering  results  suggest  that  the  low 
spatial  frequencies  prov  ide  sufficient  information  to  classify  forms.  The  results  of  such  filtering,  in  which  over  half  of 
the  higher  spatial  frequencies  are  removed  (Figure  7<>.b.  1  i  and  an  inverse  transform  reconstructs  the  pattern. are  show  n 
in  Figure  70.h.2.  Note  smoothed  triangles  with  no  intensity  biases  towards  any  orientation.  However,  filtering  has 
removed  equal  spatial  frequency  information  from  each  orientation.  The  orientation  channels  are  next  treated 
indejiendently  of  one  another. 

Further  spatial  filtering  passed  two  of  the  three  orientation  channels  shown  in  Figure  76, c. 2.  Note  that  the  triangles 
now  fsiint  in  one  direction,  appear  isosceles  in  shafie,  and  have  intensity  gradients  such  that  they  ap|icar  in  depth.  If  a 
second  combination  of  two  orientation  channels  is  filtered,  the  triangles  now  appear  pointed  in  another  direction 
<  I  igure  7<*.d.2t.  The  final  combination  of  two  orientation  channels  reveals  the  third  orientation  in  which  the  triangles 
can  appear,  the  effects  once  again  being  accompanied  by  changes  in  sha|H-  and  depth  (Figure  76,e.2l.  Thus, 
two-dimensional  filtering  by  quasi-independent  spatial  frequency  /orientation  channels  in  the  visual  system  could 
provide  a  basis  fur  the  mechanisms  behind  such  complex  three-dimensional  multistahility  . 


Some  of  the  finer  points  of  the  filtering  are  now  considered.  Can  the  triangles  be  made  to  appear  to  point  in  any  other 
directions  by  such  filtering?  The  answer  is  no.  For  example,  passing  only  one  orientation  channel  would  result  in  only 
one  side  of  each  triangle  being  reconstructed.  The  effect  would  be  to  reverse  the  contrast  of  the  edges  of  the  triangles  in 
these  pictures.  Do  the  channels  have  to  be  completely  independent  of  one  another  to  observe  such  penomena?  In  other 
words,  can  these  kinds  of  effects  require  that  the  two  channels  that  are  filtered  not  contain  energy  from  the  third?  The 
answer  is  no.  Figure  76,f,2  shows  the  information  that  is  contained  in  the  three  orientation  channels,  where  one  is  less 
filtered  out  than  the  other  two.  Note  the  isosceles  shape  of  the  triangles.  Can  such  filtering  extract  similar  orientation- 
specific  information  for  one  or  several  triangles  without  affecting  the  other?  The  answer  again  is  no.  Although  such 
filtering  could  create  slight  variations  in  the  intensity  distribution  from  one  triangle  to  another,  as  seen  from  these 
results,  their  spatial  information  can  be  acted  upon  only  as  a  whole  (this  point  will  be  expanded  later).  These  several 
results  appear  to  be  a  very  important  correlate  to  perception.  The  visual  system  appears  unable  to  manipulate  these 
objects  any  more  or  less  than  the  filtering. 

As  Attneave  ( 1%8|  noted,  similar  triangles  and  diamonds  require  special  pattern  configurations  in  order  for  them  not  to 
ap|>ear  as  a  group  that  shifts  orientation  in  unison.  If  the  triangles  are  arranged  in  collinear  units,  such  as  shown  in 
Figure  77. a,  1 .  then  subgroups  of  the  triangles  appear  to  group  independently  of  one  another.  Furthermore,  the 
triangles  within  a  particular  subgroup  tend  to  orient  themselves  along  a  common  axis. 

Vi  hat  could  be  the  basis  for  the  common  axis  in  each  group?  One  possibility  is  the  axis  formed  by  the  low  spatial 
frequencies  of  the  objects.  This  is  shown  in  Figure  77.a.2.  These  results  suggest  that  the  filtered  image  formed  by  the 
low  spatial  frequencies  could  provide  a  common  axis  at  different  orientations  on  which  the  subgroups  arrange 
themselves.  A  similar  analysis  was  used  for  Figure  77,b,l.b,2. 

Another  correlate  between  the  filtered  triangles  and  perception  is  that  of  depth.  The  intensity  distributions  of  the 
filtered  triangles  convey  a  sense  of  depth  by  appearing  to  point  out  of  the  page.  Since  the  filtering  is  a  linear  process,  it 
may  be  possible,  with  further  experiments,  to  directly  relate  the  depth  conveyed  from  the  intensity  distribution  to  the 
spatial  frequencies  and  orientation  passed  by  the  filter.  The  relationship  between  intensity  distribution  and  depth  found 
with  filtered  images  of  multistable  objects  is  the  same  relationship  that  was  found  for  the  monocular  texture  gradient 
discussed  in  Section  1 1  .<>. 

This  analysis  is  not  without  predictions.  Similar  filtering  of  scalene  triangles  required  more  critical  orientation 
selectivity  of  the  filter.  The  results  of  filtering,  shown  in  Figure  78.  are  consistent  with  the  psychophysical  data 
discussed  in  Section  (>.2  that  the  orientation  selectivity  in  the  human  visual  system  has  an  effective  half-width  of  about 
±  I .')  degrees.  The  psychophy  sical  data  suggest  that  scalene  triangles  having  an  acute  angle  of  about  1 7>  degrees  or  less 
fluctuate  much  less  readily  than  equilateral  triangles.  Those  results  further  suggest  that  triangles  will  fluctuate  much 
less  readily  as  the  shape  of  the  triangles  is  changed  from  equilateral  to  scalene  in  a  series  of  decreasing  ranges  of  the 
acute  angles. 

Finally,  these  filtering  results  predict  that  adaptation  to  gratings  that  are  collinear  with  any  side  of  the  triangles  should 
adapt  the  orientation  channels  in  the  visual  sy  stem  tuned  to  that  orientation,  thus  increasing  the  tendency  of  the 
triangles  to  point  in  the  direction  orthogonal  to  the  adapted  side.  This  prediction  was  tested  and  confirmed  on  several 
subjects  using  both  binocular  and  interocular  adapting  and  testing  conditions.  Binocular  adaptation  produced  stronger 
effects  than  did  monocular  adaptation.  Furthermore,  since  the  effect  was  found  with  the  technique  of  interocular 
transfer  (that  is,  adaptating  one  eye  and  testing  the  other),  the  fluctuation  is  not  simply  a  result  of  after-images.  This 
result  suggests  that  orientation-selective  cortical  mechanisms  underlie  the  multistability  of  such  patterns.  The  reader  is 
inv  ited  to  confirm  these  results  by  adapting  to  the  patterns  in  Figure  7.V  (Jeorgeson  I  l’Cbl  also  found  that  adapting  to 
gratings  can  affect  the  |>erception  of  multistable  triangles. 

Future  experiments  are  planned  to  [terform  more  detailed  investigation  of  the  effect  of  the  spatial  frequent')  and 
orientation  of  the  gratings  on  such  multistable  patterns. 

18.7  Summary 

The  results  reported  in  this  chapter  extend  the  application  of  the  spatial  filter  concept  front  static  to  certain  dynamic 
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changes  in  perception.  Seemingly  unrelated  perceptual  phenomena  were  related  in  common  terms  —  spatial  frequency 
and  orientation.  These  results,  in  particular  those  concerning  the  multistable  triangles,  will  be  used  to  argue  for  an 
analogy  between  the  transform  and  filtering  that  have  been  used  and  the  way  the  visual  system  appears  to  process 
spatial  objects  (see  Section  19.9.3. 1 1. 

Before  a  general  discussion  of  these  results,  one  more  investigation  is  discussed.  All  of  the  spatial  filtering  presented 
thus  far  has  considered  the  spatial  information  in  either  the  overall  MTF  or  one  bandwidth  for  each  object.  If  the  visual 
system  has  a  range  of  different  channels,  then  what  is  the  full  range  of  information  that  is  contained  in  the  channels? 
The  investigation  of  that  information  is  presented  in  the  next  chapter. 
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CHAPTER  14 


SPATIAI.  INFORMATION  FROM  A  BANK  OF  CHANNELS 


14.1  Introduction 

Most  of  the  filtering  discussed  in  the  previous  sections  concentrated  on  the  effects  of  the  overall  MTF  and  the 
information  contained  in  limited  bands  of  spatial  frequencies  of  objects.  Certain  differences  in  the  shape  and 
bandwidth  of  v  arious  spatial  filters  were  found  to  produce  little  spatial  differences  in  the  filtered  images.  However,  the 
visual  system  contains  mechanisms  tuned  to  different  ranges  of  spatial  frequency.  This  section  examines  more  closely 
the  effects  of  different  shapes  and  bandwidths  and  the  information  contained  in  different  bands  of  spatial  frequencies. 

In  particular,  a  complex  object  is  filtered  using  a  bank  of  spatial  filters  having  ideal  band-pass  and  "channel 
weighting  functions  tuned  to  different  ranges  of  spatial  frequency. 

14.2  Methods 

Both  ideal  band-pass  filters  and  filtering  based  on  the  tuning  characteristics  of  channels — the  double  exponential 
function  that  was  found  from  experimental  data  (see  Section  8.2.1 1 — were  used  for  weighting  functions.  The  bandwidth 
of  these  spatial  filters  was  obtained  from  experimental  data — one  and  two  octaves  at  half  amplitude  (see  Section  6.1 1. 
The  filters  are  isotropic;  creating  spatial  filtering  for  all  the  orientation  channels  would  result  in  too  large  a  set  of  data  to 
be  practical  at  this  time. 

The  complex  object  that  was  filtered  is  the  portrait  of  Figure  28,a.  The  main  reason  for  using  the  same  portrait  was  to 
determine  if  the  different  image  processing  systems.  IAF  and  SEMPER,  using  different  input  and  output  devices 
would  produce  similar  results. 

The  one-  and  two-octave  bandwidth  filters  were  centred  at  octave  intervals  of  1. 2.  4, 8.  16.  32.  <>4  and  128  c/f.  Thus, 
the  one-  and  two-octave  filters  represent  a  discrete  band  of  overlapping  channels  whose  centre  frequency.  fc.  is 
separated  by  one  octave.  A  one-octave  band-pass  filter,  centred  a  half-octave  higher  than  those  centre  frequencies,  was 
used  to  determine  how  critical  the  centre  frequency  was  in  providing  certain  spatial  information  contained  in  a  complex 
object. 

I’he  original  portrait  ( Figures  76-8.1, a  I  was  made  up  of  256  by  256  pels  and  contained  160  grey  levels.  The  picture 
values  at  the  border  of  the  [wirtrait  were  tapered  using  five-line  average  taper  to  reduce  spectral  energy  due  to  edge 
effects.  A  thin,  vertically  tapered  line  along  the  left  side  of  the  portrait  was  inserted  as  a  control  for  the  effects  of 
filtering  in  the  high  frequency  regions.  It  will  be  shown  that  the  high  spatial  frequencies  contain  so  little  energy  that  the 
line  was  needed  to  convince  the  reader  that  the  original  portrait  was  filtered. 

The  spatial  filtering  was  done  as  described  in  Section  3.2.  The  amplitude  values  of  the  reconstructed  portraits  were  not 
normalized  and  thus  reflected  the  actual  amount  of  energy  in  the  spectral  bandwidth  that  was  passed  by  each  filter. 
Every  attempt  was  made  to  process  the  film  and  prints  in  the  same  manner.  However,  with  such  a  large  set  of  pictures, 
some  variation  was  unavoidable.  I  he  slight  variations  do  not  affect  the  interpretations  of  the  results. 

14.3  ( Comparing  Band-Pass  and  Channel  F  iltering  of  a  Portrait 

The  one-octave  ideal  band-pass  filter  t  Figure  76)  greatly  reduced  the  amount  of  general  form  or  detailed  information 
altoiit  the  face.  This  lack  of  information  is  especially  apparent  when  a  comparison  is  made  between  these  results  and 
information  contained  in  the  two-octave  band-pass  filtered  portraits  shown  in  Figure  80.  The  hint  of  a  face  in  the 
one-octave  filtered  [>ortrait  show  n  in  F’igure  76, d  is  clearly  captured  by  the  two-octave  filter  shown  in  F'igure  80. d.  Note 
that  the  two-octave  filter  centred  at  16  c/f  provides  almost  all  the  information  about  the  portrait  except  for  the  fine 
details.  The  details  are  found  in  the  (Hirtraits  filtered  from  higher  spatial  frequencies. 

Th»  two-octave  band-pass  filtered  portraits  contain  considerably  more  information  than  do  the  one-octave  band-pass 
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filtered  objects.  Could  the  centre  frequency  be  too  low  for  the  one-octave  band-pass  filter  to  be  able  to  pass  that 
increased  information?  This  is  unlikely  because  the  one-octave  band-pass  filtered  portrait  centred  a  half-octave  above 
the  centre  frequency  used  previously  has  little  gain  in  information,  as  shown  in  Figure  hi .  The  basic  form  of  the  face  is 
reduced  in  Figure  hi  ,d.  There  is  a  slight  increase  in  the  amount  of  information  contained  in  Figure  hi  ,f.  but  even  this 
increase  does  not  provide  as  much  information  as  Figure  hO.f.  There  are,  of  course,  many  other  possible  centre 
frequencies  that  might  provide  a  little  more  information  about  the  original  portrait  than  the  two  used  here.  However,  it 
is  highly  unlikely  that  a  shift  in  centre  frequency  for  the  one-octave  filter  could,  in  general,  provide  the  amount  of 
information  for  the  portrait  that  was  passed  by  the  two-octave  filter. 

There  is  little  information  about  the  portrait  in  the  high  spatial  frequencies,  as  shown  in  Figures  79-80.  h.  The  highest 
spatial  frequencies  in  the  last  quarter  of  the  spectrum,  %  to  128  c/f,  contain  virtually  no  information.  Only  the  thin 
control  line  is  visible  (Figure  81.il.  Similar  results  using  the  one-  and  two-octave  channel  filters  are  shown  in  Figures 
82  and  88. 

14.8.1  General  Discussion 

Note  that  the  periodic  ripples  that  were  present  in  the  ideal  band-pass  filtered  portraits  are  gone  in  the  channel  filtered 
portraits.  However,  note  that  the  ripples  on  both  sides  of  any  edges  in  the  ideal  band-pass  filtered  portraits  can  still  be 
seen  in  the  channel  filtered  portraits.  The  absence  of  the  periodic  ripples  in  the  channel  filtered  portrait  is  due  to  the 
positive  and  negative  exponential  taper  of  the  channel  filter.  Any  taper  of  the  abrupt  cutoff  of  an  ideal  band-pass  filter 
will  reduce  the  periodic  ripples  (e.g.,  see  Brigham,  1974 1.  The  particular  distribution  of  the  light  and  dark  ripples  is  due 
to  the  finite  bandwidth  and  shape  of  the  filters.  The  line-spread  function  of  this  type  of  filter  was  shown  in  Figure  4.b  to 
have  positive  and  negative  weighting,  i.e.,  ‘‘lateral  inhibition."  The  square-wave  response  was  shown  in  Figure  4,c. 
Thus,  the  light  and  dark  ripples  at  the  edges  of  these  filtered  portraits  are  the  kind  of  contrast  changes  that  can  be 
expected  in  spatial  objects  from  the  effects  of  spatial  filters  producing  lateral  inhibition. 

Note  that  the  filtering  that  produced  the  effects  of  lateral  inhibition  has  not  produced  edge  sharpening.  There  are  no 
edges  in  any  filtered  portrait  that  are  seen  as  sharper  than  the  same  edges  in  the  original  portrait.  The  effect  of  filtering 
due  to  lateral  inhibition  is  to  increase  the  contrast  of  edges,  i.e. ,  to  enhance  the  edges  b_v  reducing  the  contrast  away 
from  the  edges. 

The  increased  contrast  of  the  edges  is  similar  to  the  contrast  distribution  seen  from  Mach  bands  (see  Ratliff.  1%8: 
Patel.  1966;  Cornsweet.  1  <>7(H.  Since  the  spatial  frequencies  of  any  figure  that  produces  Mach  bands  w  ill  be  filtered 
differently  by  the  MTF  as  viewing  distance  changes,  and  Mach  bands  remain  over  relatively  large  ranges  of  view  ing 
distance,  it  is  unlikely  that  Mach  bands  can  result  from  any  one  filter.  However,  assume  that  each  channel  filter  has  the 
same  filter  shape.  As  the  object  that  produces  a  Mach  band  is  viewed  from  different  distances,  it  would  be  filtered  by 
different  channels  with  similar  filter  sha/H's  audits  appearance  would  not  change  until  the  viewing  distance  became 
very  close  or  very  far. 

The  reader  is  encouraged  to  compare  the  general  information  about  the  face  of  the  two-octave  band-pass  filtered 
portraits  of  Figures  80,  88. d  with  the  MTF  low-pass  filtered  portraits  of  Figure  28,a,b.e.8,4.  The  prev  ious  filtered 
(Hirtraits  were  done  with  the  square-shaped  spatial  filter  ov  er  the  same  bandw  idth  as  the  present  filtering.  The  general 
information  about  the  face  in  all  these  filtered  portraits  is  very  similar.  These  results  further  suggest  that  spatial  filters 
hav  ing  different  weighting  functions  can  provide  similar  spatial  information  about  objects.  This  particular  result  is 
important  la-cause  it  means  that  many  facets  of  perception  can  be  investigated  with  a  generalized  set  of  spatial  filters 
without  great  concern  about  the  precise  shape  or  bandwidth  of  the  actual  channels  in  the  v  isual  system. 

The  major  difference  between  the  filtered  portraits  resulting  from  the  different  spatial  filters  is  the  contrast.  Thus,  the 
particular  shape  and  bandwidth  of  the  channels  will  most  likely  be  important,  primarily  for  those  perceptual  correlates 
that  require  more  precise  contrast  information,  e.g.,  the  contrast  of  Mach  bands.  The  sha|M-  and  bandwidth  of  the 
channel  filters  is  also  probably  important  for  refining  measurements  of  lines  and  angles  of  geometric  illusions  and 
defining  the  resolution  of  objects  under  threshold  conditions  to  relate  the  filtered  iniages  to  experimental  data. 


The  fidelity  of  the  filtered  portraits  in  prov  iding  the  spatial  information  actually  present  in  the  original  |>ortrait  is  easily 
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tested.  The  reader  should  view  the  two-octave  channel  filtered  portraits  from  a  distance  sufficient  to  make  the  oritnnai 
|H>rtrait  just  a  grey  patch.  At  that  distance,  the  original  portrait  will  look  similar  to  the  first  filtered  porti  a>i  containing 
only  the  very  low  spatial  frequencies  (fc  =  1 1  because  the  visual  system  is  acting  like  a  loo  -pass  filter.  Slowly  moving 
closer  results  in  the  original  portrait  looking  like  the  filtered  portrait  fiom  the  second  channel  (fc  =  21.  Moving  closer 
still  will  provide  a  good  agreement  between  the  original  portrait  and  the  filtered  portrait  from  the  third  channel  (fc  = 

41.  As  one  moves  even  closer,  a  difference  in  contrast  will  be  noticed  because  the  original  portrait  will  be  seen  with  the 
combined  channels  whose  individual  contributions  are  shown  separately  in  the  channel  filtered  portraits.  However, 
each  higher  channel  provides  different  amounts  of  resolution.  The  good  agreement  of  the  degree  of  resolution  seen 
between  the  original  and  the  filtered  portraits  will  be  seen  as  one  moves  closer  still. 

14.2. 1.1  Information  Contained  in  a  Hierarchy  of  Filtered  Images 

Note  the  different  information  about  the  portrait  that  can  be  found  in  each  of  the  seven  two-octave  bandwidth  channels 
of  Figure  83.  The  existence  of  an  object  is  evident  from  the  large  regions  of  different  contrast  in  the  first  channel  lfc  = 

1 1.  The  gross  elliptical  shape  of  an  object  that  appears  right  side  up  is  seen  in  the  seond  channel  (fc  =  21.  The  third 
channel  lfc  =  41  definitely  provides  enough  information  to  classify  the  object  as  a  face.  The  identification  of  the  face 
needs  a  little  more  information.  The  fourth  channel  (fc  =  8)  suggests  that  the  face  is  that  of  a  woman  from  the  hair 
style,  etc.  Identification  would  seem  possible  given  a  limited  set  of  similarly  filtered  portraits  to  choose  from.  Clearly, 
the  face  in  the  fifth  channel  (fc  =  161  is  the  same  as  that  of  the  original  channels;  and  identification  could  present  no 
problem,  given  any  large  set  of  other  portraits.  If  information  about  the  details  of  the  portrait  is  needed — e.g.,  the  hair 
across  her  forehead,  the  texture  of  her  hair,  size  of  the  pupils,  outline  of  her  lips — then  that  information  is  found  in  the 
two  remaining  channels  (fc  =  32,  641.  (These  last  two  channels  are  also  needed  to  see  the  control  line.  I  Thus,  these 
seven  channels  provide  the  full  spectrum  of  information  that  is  needed  for  any  perceptual  task. 

It  is  interesting  to  note  that  central  vision  contains  an  array  of  about  200  by  200  cones.  Requiring  two  cones  to  sample 
the  high  spatial  frequency  results  in  100  spatial  harmonics.  This  portrait  contains  128  two-dimensional  harmonics.  This 
resolution  would  allow  only  up  to  about  7  channels  whose  centre  frequencies  were  separated  by  one  octave. 

These  results  also  suggest  a  possible  reason  for  having  inhibition  between  spatial  frequency  channels  (Tolhurst,  10721. 

In  addition  to  sharpening  up  the  bandwidth  of  channel  filters,  inhibition  between  channels  further  reduces  the  spatial 
information  that  is  extracted  from  an  object.  Inhibition  between  channels  could  reduce  the  number  of  channels  needed 
to  provide  relevant  information  about  an  object.  If  there  were  complete  inhibition  between  channels  over  a  two-octave 
bandwidth,  then,  for  a  two-octave  bandwidth  channel,  only  information  contained  in  the  channels  with  centre 
frequencies  1.4,  16  and  64  would  be  processed.  The  filtered  portraits  from  these  four  channels  can  be  seen  to  contain 
sufficient  information  for  the  existence  lfc  =  1 1,  general  form  of  a  face  <fc  =  4).  identification  of  a  face  (fr  =  161  :  ?/d 
details  of  the  face  I fc  =  641.  If  the  inhibition  is  less  strong  between  channels,  then  seven  overlapping  two-octave-wide 
channels  would  be  possible.  Thus,  by  having  inhibition  between  channels,  the  spatial  information  required  for  relevant 
perceptual  tasks  can  be  separated  into  4  to  7  semi-discrete  channels. 

In  summary,  this  portrait,  filtered  using  the  same  shape  and  bandwidth  filter  as  that  found  in  a  wide  variety  of 
neurophysiological  and  psychophysical  studies,  demonstrates  the  amount  and  kind  of  spatial  information  that  can  be 
contained  in  banks  of  quasi-discrete  channels  in  the  visual  system  (assuming  linearity,  of  course  I.  These  demonstrations 
further  suggest  that  the  particular  shape  and  bandwidth  are  not  critical  in  conveying  similar  spatial  information  about 
objects  in  similar  region  of  spatial  frequency.  Seven  octaves  of  overlapping  channels  centred  one  octave  apart  provide  a 
lexicon  of  spatial  information  in  a  hierarchy  of  filtered  images  about  the  existence,  the  basic  form  and  identification, 
and  resolution  of  the  finest  detail  of  a  complex  object. 
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CHAPTER  15 


GENERAL  DISCUSSION  AND  SUMMARY 
OF  THE  RESULTS  OF  SPATIAL  FILTERING 


15. 1  The  Classification  of  Simple  and  Complex  Objects 

A  wide  variety  of  objects  have  been  spatially  filtered.  A  narrow  band  of  one  to  two  octaves  (about  three  harmonics)  of 
low  spatial  frequencies  provided  similar  generalized  form  information  from  discriminably  different  objects  of  the  same 
class.  These  results  demonstrated  the  Gestalt  concept  that  the  whole  is  different  from  the  sum  of  the  parts.  Similar 
results  were  obtained  with  a  complex  object — a  portrait.  Again  the  low  spatial  frequencies  provided  general  forms  from 
a  wide  variety  of  discriminably  different  portraint  configurations. 

Experimental  evidence  supported  the  notion  that  low  spatial  frequencies  provide  sufficient  information  for  the 
classification  of  an  object.  Ginsburg  (1971  ,a,bl  showed  that  subjects  could  learn  random  letter  arrays  which  contained 
only  low  spatial  frequencies  at  the  same  rate  and  with  the  same  success  as  the  original  high  spatial  frequency  letters. 
Woodhouse  and  Campbell  ( 1977)  have  shown  that  four  harmonics  are  sufficient  to  provide  face  recognition  with  a  small 
number  of  faces.  Part  IV  of  this  dissertation  will  provide  evidence  that  an  experiment  done  hundreds  of  times  each  day 
throughout  the  world — visual  testing  with  the  Snellen  letter  acuity  chart — supports  the  suggestion  that  a  small  number 
of  spatial  frequencies  are  sufficient  for  the  classification  of  objects. 

15.2  Visual  Textures 

The  initial  studies  showed  that  filtering  by  the  anisotropic  MTF  only  reduced  the  detail  of  objects.  The  next  study  w  ith 
visual  textures  showed  that  the  anisotropy  of  the  MTF  segregated  objects  that  differed  only  in  orientation.  Certain  first 
and  second  order  problems  with  texture  arrays  and  random  dot  patterns  appeared  explicable  in  terms  of  the  filter 
concept.  The  perception  of  a  monocular  texture  gradient  also  had  a  correlate  to  the  filtered  image.  The  knowledge 
gained  from  the  initial  studies  of  texture  was  used  to  extract  complex  forms  from  a  real  scene. 

15.5  V  isual  Illusions 

Visual  illusions  were  investigated  next.  The  MTF  and  low-pass  filters  provided  distortions  of  filtered  images  that 
corresponded  well  to  the  distortions  that  were  perceived.  It  seems  no  coincidence  that  the  most  robust  distortions  and 
the  best  results  of  an  experiment  that  tested  the  filter  theory  were  found  with  spatial  filters  whose  bandwidth  was  the 
same  as  that  which  provided  general  form  information  in  the  demonstrations  of  the  previous  chapters.  Only  one  visual 
illusion,  the  Zdllner  illusion,  did  not  yield  to  a  simple  explanation  by  spatial  filtering. 

A  relationship  was  suggested  between  eye  scans  and  certain  regions  of  contrast  of  the  filtered  image  of  visual  illusions. 
The  anisotropy  of  the  M'FF  offered  an  explanation  of  why  certain  geometric  distortions  change  on  rotation. 

1 5.1  Multistable  Objects 

Certain  multistable  objects  were  examined  next  in  which  many  different  alternating  perceptions  were  unified  by  the 
concept  of  filtering  by  spatial  frequency  /orientation  channels.  An  explanation  of  some  multistable  objects  was  given 
from  the  difference  in  size  of  the  objects  in  one  scene.  Other  multistable  objects  considered  the  segregation  problem  and 
how  filtered  images  could  favour  different  areas  for  analysis.  Explanations  of  still  other  multistable  objects  suggested 
selective  filtering  of  specific  orientation  channels. 

15.5  V  Hierarchy  of  Filtered  Images 

Finally  ,  a  detailed  study  on  a  portrait  compared  ideal  band-pass  filters  with  biologically  derived  channel  filters.  The 
results  of  the  study  showed  little  difference  between  the  forms  from  the  two  different  kinds  of  filters.  However,  a  bank 
of  two-octave- wide  filters  provided  more  information  in  almost  all  the  images  than  did  the  one-octave-w  ide  filters. 


Seven  filtered  images  created  from  two-octave-wide  channels  provided  information  about  the  portrait  that  ranged  from 
the  basic  form  and  recognition  to  the  finest  detail. 

This  hierarchy  of  filtered  images  is  suggested  as  the  kind  of  spatial  information  that  is  available  to  an  observer.  It  helps 
answer  the  following  question. 

11. (>  Why  Don't  We  See  the  Basic  Forms  of  Objects? 

An  important  point  to  be  considered  is:  Why  are  discrete  pattern  elements  observed  in  detail,  such  as  the  dots  of  a  dot 
G.  and  not  just  the  low  spatial  frequencies  that  have  been  shown  to  contain  the  relevant  form  information?  One 
explanation  is  that  there  is  a  hierarchy  of  filtered  images,  each  of  limited  bandwidth.  Although  there  is  an  ever 
increasing  amount  of  evidence  that  mechanisms  in  the  visual  system  filter  the  intensity  distribution  of  the  retinal  image 
using  quasi-independent  spatial  frequency /orientation  channels,  limited  and  unstable  manipulation  of  such 
information  can  be  seen  only  under  somewhat  special  and  isolated  conditions  (monocular  rivalry,  for  example).  If  the 
visual  system  could  isolate  the  information  from  any  different  spatial  frequency  channel  independently  of  all  the  others 
at  the  early  stages  of  processing,  then  the  G  form  could  be  seen.  But  “seeing"  the  inhumation  in  all  the  different  spatial 
frequency  bands  would  result  in  chaos  for  perception.  Objects  would  be  seen  at  will  varying  in  resolution  from  diffuse 
forms  to  mere  outlines.  What  appears  to  be  seen,  in  general,  is  the  detail  in  objects  from  the  activity  of  all  the  channels 
limited  only  by  the  resolution  lost  by  the  filtering  of  the  early  stages  of  the  visual  system,  as  was  demonstrated  bv  the 
MTF  spatial  filters. 

If  a  parallel  set  of  filtered  images  is  assumed  to  exist,  then  it  follows  that  another  stage  of  visual  processing  may  attend 
specifically  to  the  image(s)  that  contain  the  low  spatial  frequencies  of  the  form.  One  channel  would  provide  an  image 
that  contained  generalized  form  information  which  was  somewhat  independent  of  the  channels  that  contained  the 
higher  spatial  frequency  images.  Since  that  low  spatial  frequency  channel  has  been  demonstrated  to  provide  sufficient 
information  for  identification,  it  might  provide  the  input  to  memory.  This  processing  scheme  could  provide  the  basis  for 
classification  of  objects,  despite  small  discriminate  differences.  It  may  also  offer  a  coherent  explanation  of  why  objects 
are  correctly  classified  at  distances  at  which  only  the  low  spatial  frequencies  are  passed  by  the  visual  system  as  well  as  at 
closer  distances  w  here  the  details  of  the  pattern  are  seen. 

1.1.7  Predicting  the  Identification  of  Complex  Objects  by  the  Filtering  Characteristics  of  the  Human 
V  isual  System 

This  part  of  the  dissertation  presented  a  filter  concept  that  offers  possible  explanations  of  a  wide  variety  of  visual 
phenomena.  The  preceding  filtering  demonstrated  that  a  single  channel  with  a  bandwidth  of  one  to  two  octaves  passed 
sufficient  spatial  information  for  the  classification  and  identification  of  many  objects.  Can  this  information  be  used  to 
predict  the  identification  of  complex  objects  by  the  filtering  characteristics  of  the  human  visual  system?  One  obvious 
experiment  to  help  answer  this  question  is  the  removal  of  a  one-  to  two-octave  spatial  frequency  bandw  idth  which  was 
show  n  to  be  important  for  the  classification  and  identification  of  the  object  and  to  determine  if  performance  was 
degraded  in  some  specified  way.  The  main  problem  with  this  approach  is  that  it  assumes  that  mechanisms  in  the  visual 
system  behave  as  perfect  filters — that  high  spatial  frequencies  which  are  removed  are  not  replaced  by  some  mechanism 
in  the  visual  system.  'Phis  assumption  cannot  be  justified  for  several  reasons.  First,  as  mentioned  in  Section  1.2. 1 .  the 
fact  that  the  sizes  of  retinal  receptive  fields  change  with  retinal  location  means  that  the  initial  sampling  of  the  retinal 
image  is  nonuniform.  Unless  the  filtered  image  is  first  prefiltered  with  space  variant  filtering  characteristics  of  the 
retina,  the  retinal  image  will  be  filtered  in  an  unspecified  manner.  Second,  the  retinal  image  is  not  static.  The  ey  e  is  in 
constant  motion  due  to  voluntary  and  involuntary  saccades.  The  motion  will  smear  the  retinal  image  in  a  complex 
fashion.  For  example,  the  sharp  edges  of  a  high-pass  filtered  image  will  suffer  some  degree  of  smoothing  at  the  retina. 
The  effect  of  eye  movements  may  be  reduced  by  short  presentation  times  in  a  tachistoscope.  However,  that  would 
introduce  additional  filtering  by  the  temporal  properties  of  the  visual  system.  Stabilizing  the  image  would  cause  it  to 
disappear  altogether.  Third,  Campbell.  Howell,  and  Hobson(  1*171 1  showed  that,  at  low  spatial  frequencies,  the  higher 
harmonics  can  generate  the  ap|>earance  of  a  square-wave  grating  even  when  the  fundamental  component  has  been 
removed.  The  missing  fundamental  has  been  "filled  in."  This  filling-in  phenomenon  also  lends  to  occur  between  sharp 
edges  of  other  patterns  (O'Brien,  I ‘).1H;  Craik.  I'M);  Cornsweel,  l()70l.  Until  this  filling-in  mechanism  is  understood 
and  quantified,  it  will  be  assumed  that  edges  of  objects  provide  sufficient  information  for  the  creation  of  certain  low 


spatial  frequencies.  For  example.  Figure  31  ,c  showed  that  the  edges  of  a  portrait  also  contained  low  spatial  frequencies. 
The  outline  subjective  contour  of  Figure  64,a,  1  also  had  low  frequency  components.  In  sum,  although  it  is 
operationally  easy  to  remove  low  spatial  frequencies  from  objects,  this  provides  no  guarantee  that  those  spatial 
frequencies  will  not  be  replaced  to  some  degree  by  mechanisms  in  the  visual  system  and  that  they  will  not  generate  other 
spatial  frequencies.  Although  these  caveats  suggest  caution  in  interpreting  experiments  in  which  bands  of  spatial 
frequencies  are  removed,  the  author  has  attempted  such  an  analysis  with  some  success  with  an  analysis  of  the  Kanizsa 
triangle  (see  Section  12.7.11. 

On  the  other  hand,  rather  than  filtering  objects  to  test  the  hypothesis,  the  filtering  characteristics  of  the  visual  system 
could  be  modified  to  reduce  sensitivity  over  selected  ranges  of  spatial  frequency  using  adaptation  techniques  (Section 
6.1.1 1.  Further,  masking  techniques  (Section  6. 1 .21  could  be  used  to  reduce  visibility  of  certain  ranges  of  spatial 
frequencies  of  objects. 

Another  possibility  is  to  use  abnormal  filter  characteristics.  For  example,  if  a  subject  had  an  abnormal  visual  condition 
in  which  only  one  channel  functioned,  then  one  hypothesis  is  that  the  subject  would  be  able  to  identify  only  those 
objects  at  the  limited  range  of  distances  over  which  the  relevant  low  spatial  frequencies  of  the  object  fell  within  the 
bandw  idth  of  that  channel.  Such  a  subject  who  had  what  appeared  to  be  only  a  3  c/d  channel  due  to  keratoconus  (a 
disease  affecting  the  shape  of  the  cornea  I  was  reported  by  Ohzu  and  Kawara  ( 1 6751.  Unfortunately,  they  reported  no 
further  investigation  of  this  patient  due  to  the  rapid  decline  of  his  vision. 

There  are,  however,  other  abnormal  visual  conditions  which  result  in  selective  losses  in  contrast  sensitivity.  For 
example,  Bodis-Wollner  ( 1672)  investigated  the  contrast  sensitivity  of  16  patients  with  cortical  lesions.  He  found  that 
the  greatest  loss  in  contrast  sensitivity  occurred  at  the  high  spatial  frequencies.  One  subject  suffering  from  a 
meningioma  pressing  on  the  left  occipital  lobe  had  a  mid-frequency  loss  from  about  4  to  12  c/d.  Selective  losses  in 
contrast  sensitivity  at  some  spatial  frequencies  of  amblyopes  have  also  been  found  in  studies  reported  by  ltoi,  Kato. 
Sugimachi,  Kawamura.  Ohzu,  and  Kawara  ( 16751. 

These  selective  losses  in  sensitivity  to  some  spatial  frequencies  could  be  used  to  test  the  hypothesis  that  a  one-  to 
two-octave  band  of  spatial  frequencies  of  objects  is  required  for  their  identification.  Assume  that  mechanisms  in  the 
visual  system  cannot  perfectly  replace  any  high  spatial  frequency  information  that  is  removed  by  filtering.  This 
dissertation  has  presented  many  examples  of  low-pass  filtered  images  whose  brightness  distribution  ap(>eared  very 
similar  to  that  of  the  original  object  w  hen  viewed  at  the  appropriate  distance  such  that  the  visual  system  low-pass 
filtered  the  original  object  to  the  same  degree  as  the  filtered  image.  Thus,  unlike  the  high-pass  filtered  images,  there  is 
no  evidence  that  the  visual  system  creates  high  spatial  frequencies  from  low-pass  filtered  images  to  any  high  degree.  If 
high-pass  spatial  filtering  removes  bands  of  spatial  frequencies  that  remain  relevant  for  identification,  and  those  spatial 
frequencies  that  remain  fall  within  a  range  of  decreased  visual  sensitivity,  then  reduced  identification  would  he 
predicted. 

Another  experiment  could  also  be  used  to  test  the  same  hy!>othesis;  namely,  the  reduced  identification  of  objects  could 
be  related  to  a  variety  of  contrast  sensitivity  functions  having  reduced  sensitivity  over  different  ranges  of  spatial 
frequencies.  Adaptation  and  masking  techniques  could  be  used  to  reduce  sensitivity.  However,  since  the  abnormal 
filter.ng  characteristics  of  subjects  previously  reported  appear  to  be  robust,  they  do  not  require  the  additional 
pro*  edures  and  assumptions  of  adaptation  and  masking;  and,  since  investigating  those  characteristics  may  provide 
additional  clues  about  vision,  abnormal  vision  was  selected  as  the  means  to  test  the  hypothesis. 

Based  on  preceding  considerations,  it  was  decided  to  study  abnormal  vision  to  find  similar  patients  with  reduced 
contrast  sensitivity  over  different  ranges  of  spatial  frequencies.  Since  research  into  abnormal  vision  was  somewhat 
outside  the  main  thrusts  of  this  dissertation,  this  research  is  appended  in  Chapters  A- 1  through  A-4.  Contrast 
sensitivity  functions  showing  losses  in  sensitivity  at  low.  middle,  and  high  spatial  frequencies  were  obtained  from 
subjects  having  amblyopia  and  multiple  sclerosis.  (Inly  one  subject  had  contrast  sensitivity  over  a  narrow  range  of 
spatial  frequencies.  Unfortunately,  illness  prevented  her  from  being  used  in  further  ex|«'rimerits.  <  >ne  subject  had  a 
narrow-band  loss — a  "notch,'  in  her  contrast  sensitivity  function  that  was  to  be  used  to  test  reduced  visual  performance 
over  predicted  ranges  of  sizes  of  objects.  Sensitivity  increased  over  that  narrow  range  of  spatial  frequencies  before  the 
experiment  was  conduc  ted.  However,  the  gain  in  sensitivity  resulted  in  an  increase  in  Snellen  letter  acuity  that  was 
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related  to  the  lower  spatial  frequencies  of  Snellen  letters,  as  predicted  (Section  18.3).  (This  interesting  result  suggested 
that  visual  performance  can  be  increased  for  certain  cases  of  amblyopia  using  grating  stimuli  (Section  A-3.5).  Although 
no  single  narrow-band  channels  or  notches  were  obtained  from  the  other  subjects,  the  number  of  subjects  tested  did 
provide  a  wide  range  of  abnormal  contrast  sensitivity  functions  used  to  test  the  hypothesis  that  a  one-  to  two-octave 
bandwidth  of  the  low  spatial  frequencies  of  Snellen  letters  can  be  used  to  predict  Snellen  letter  acuity  from  the  contrast 
sensitivity  function.  That  research,  requiring  knowledge  about  the  relationship  between  the  contrast  sensitivity 
function  and  the  identification  of  objects,  is  presented  in  the  next  three  chapters. 
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PART  IV 


RELATING  CONTRAST  SENSITIVITY  TO  THE 
IDENTIFICATION  OF  COMPLEX  OBJECTS 


“Consider  your  verdict."  the  King  said  to  the  Jury. 
"Not  yet.  not  yet!  ’’  the  Rabbit  hastily  interrupted, 
“There's  a  good  deal  to  come  before  that." 


from  Alice  in  Wonderland 


CHAPTER  1() 


SNELLEN  LETTER  ACI  ITV  ANI)  SPATIAL  FREQUENCY 


16.1  Introduction 

The  next  three  chapters  attempt  to  determine  if  the  threshold  measurements  with  sine-wave  gratings  can  predict  the 
suprathreshold  measurements  of  Snellen  letter  acuity.  This  requires  knowledge  about  the  relevant  spatial  frequencies 
that  make  up  each  Snellen  letter  and  the  minimum  (threshold)  contrast  required  for  their  identification.  Since  it  has 
been  shown  that  the  visual  system  is  linear  at  threshold  for  one-dimensional  waveforms  (Campbell  and  Robson.  1%8I 
and  two-dimensional  sinusoids  (Carlson,  et  al.,  l'ITT).  it  could  be  expected  that  Snellen  letter  acuity  can  be  predicted 
from  contrast  sensitivity  to  sine-wave  gratings. 

There  have  been  previous  attempts  to  predict  Snellen  letter  acuity  from  the  cutoff  spatial  frequency  obtained  from 
contrast  sensitivity  measurements  with  little  success  (Gstalder  and  Green.  1671;  Bodis-Wollner,  1672;  and 
Bodis-Wollner  and  Diamond.  16761.  However,  the  band-pass  filtering  presented  in  this  dissertation  has  shown  that 
many  objects  can  be  identified  correctly  from  a  range  of  one  to  two  octaves  (2  to  4  c/ol  of  low  spatial  frequencies.  Thus, 
the  relationship  between  Snellen  letter  acuity  and  contrast  sensitivity  is  determined  by  considering  the  minimum  spatial 
frequencies  of  the  letters  required  for  their  identification. 

16.2  The  Theoretical  Minimum  Number  of  Spatial  Frequencies  Necessary  for  Identification  of  Snellen  Letters 

The  spatial  frequency  components  that  make  up  each  Snellen  letter  were  determined  as  follows.  Each  Snellen  letter  was 
formed  on  a  .7  by  .7  element  square  grid.  The  lines  that  formed  each  part  of  the  letter  were  one  element  w  ide.  The  size  of 
each  letter  on  line  6  of  a  standard  Snellen  chart,  viewed  at  a  distance  of  6  meters  (6/61.  subtends  3  minutes  of  arc.  and 
the  width  of  the  lines  that  form  each  letter  subtends  1  minute  of  arc.  In  order  to  be  consistent  with  definitions  in  the 
previous  sections,  the  fundamental  spatial  frequency  of  the  letters  was  taken  as  the  reciprocal  of  the  letter  size.  Thus, 
the  fundamental  spatial  frequency  of  each  letter  on  Snellen  line  6/6  is  12  c/d. 

The  Nyquist  frequency  (see  Section  2.4)  can  be  used  to  determine  the  minimum  theoretical  sampling  limit  for  the 
spatial  information  of  the  Snellen  letters.  Since  the  Snellen  letters  are  based  on  a  7  by  7  sampling  element  grid,  the 
theoretical  minimum  number  of  spatial  frequencies  is  half  the  number  of  the  sampling  elements,  or  2.7  cycles  per  letter 
width  (c/ 1 1.  This  means  that  each  line  pair,  i.e.,  adjacent  black  and  white  lines,  of  a  Snellen  letter  will  la-  sampled  by 
one  cycle  of  sine  wave.  Hence,  the  theoretical  minimum  number  of  spatial  frequencies  required  to  form  each  Snellen 
letter  is  2.7  c/ 1  in  the  x  and  y  dimensions  and  3.7  c/1  along  the  diagonals  (i.e..  2.7  x  V2I.  The  spatial  frequencies  in  c/d 
necessary  to  pass  the  first  2.7  cycles  of  Snellen  letters  on  each  line  are  shown  in  Figure  84.  (In  keeping  with  the  previous 
analysis,  one  cycle  across  the  letter  corresponds  to  one  harmonic.  However,  since  this  analysis  deals  with  half-cycles, 
half-harmonics  is  a  meaningless  term  and  cycles  rather  than  harmonics  w  ill  be  used  in  the  subsequent  analysis.  I  Note 
that  letters  on  line  6/6  have  their  2.7  <7 1  at  30  c/d.  The  spatial  frequency  for  2.7  (7 1  for  Snellen  line  6/3 — the  limit  of 
visual  acuity — is  60  c/d. 

it  must  be  emphasised  that  this  analysis  is  not  based  explicitly  on  the  line  width  of  the  letters.  Rather,  this  analysis 
assumes  that  it  is  the  minimum  filtering  or  sampling  of  the  features  of  the  letters  that  is  important  for  visual  processing— 
an  assumption  hated  on  the  filtering  results  from  previous  sections  that  showed  that,  generally,  only  a  two-octave 
bandwidth  was  necessary  for  the  identification  of  many  objects. 

It). 3  Spatial  Filtering  of  Snellen  Letters 

Typical  Snellen  letters  were  filtered  to  see  how  many  cycles  were  actually  required  for  identification.  The  minimum 
number  of  cycles  required  for  the  identification  of  any  object  can  lx1  determined  from  the  minimum  v  iewing  distance. 
However,  the  pictures  of  the  filtered  images  provide  the  advantages  of  magnification,  which  allows  an  easier  and  more 
stable  determination  of  the  features  of  the  filtered  image  than  those  seen  at  distances  that  produce  the  same  degree  of 
filtering.  The  two  Snellen  letters  that  were  chosen  to  lx-  filtered  are  the  most  easily  identified  letter  I.  and  the  relatively 
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more  difficult  letter  E.  Those  letters,  shown  in  Figure  85,  were  synthesized  by  low-pass  filtering  from  0.5  to  3  c/I  in 
half-cycle  steps  using  the  techniques  previously  discussed  in  Sections  4.2.1  and  4.2.2.  The  contrast  of  all  the  letters  is  76 
percent.  Note  that  the  L  required  only  up  to  and  including  1.5  c/1  for  identification  whereas  the  E  requires  2.5  c/I. 
These  results  suggest  why  the  L  has  been  found  to  be  more  visible  than  the  E  in  Snellen  acuity  (Sheard,  1922; 
Lebensohn,  1965). 

Sufficient  information  for  the  identification  of  the  L  and  E  can  also  be  shown  to  exist  in  band-pass  regions  of  2.5  cycles 
from  1.5  to  3  c/1  (Figure  85, a, b, 7)  and  2  cycles  from  2  to  3  c/1  (Figure  85,a,b,8l.  However,  the  contrast  of  each  letter 
is  reduced  relative  to  the  surround,  and  each  is  clearly  less  visible  in  comparison  to  the  low-pass  filter  letters  when 
viewed  at  a  distance.  Thus,  for  the  present,  it  will  be  assumed  that  the  relevant  spatial  frequencies  for  the  identification 
of  Snellen  letters  are  from  1 .5  to  2.5  c/1 . 

Another  point  to  be  emphasized  is  the  lack  of  need  of  precise  phase  information  by  filtering  mechanisms  to  capture  the 
basic  form  of  an  object.  There  is  little  difference  between  the  filtered  Ls  from  1 .5  to  3  c/ 1  and  the  Es  with  2.5  and  3  c / 1 . 
even  though  the  individual  lines  that  form  the  letters  are  being  filtered  by  spatial  frequencies  that  differ  in  position 
(phasel  relative  to  the  lines  of  each  letter.  These  points  are  important  to  an  understanding  of  how  objects  can  be 
identified  without  critically  matched  templates,  features,  or  receptive  fields. 

It  is  clear  from  these  filtering  results  that  1 .5  to  2.5  c/ 1  can  provide  sufficient  information  for  the  identification  of 
Snellen  letters.  The  next  piece  of  information  that  is  needed  in  order  to  relate  Snellen  letter  acuity  to  contrast  sensitivity 
is  the  contrast  of  the  Snellen  letters  required  for  their  identification.  This  is  obtained  in  the  next  chapter. 
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CHAPTER  17 


THE  MINIMUM  CONTRAST  REQUIRED  FOR  THE 
DETECTION  AND  IDENTIFICATION  OF  SNELLEN  LETTERS 


17.1  Introduction 

Knowledge  of  both  the  minimum  contrast  and  minimum  number  of  spatial  frequencies  can  provide  a  pair  of  numbers 
that  can  be  used  to  determine  the  minimum  information  required  for  the  identification  of  Snellen  letters.  The  previous 
chapter  presented  evidence  that  about  1 .5  to  2.5  cycles  per  letter  is  sufficient  for  the  identification  of  Snellen  letters. 
This  experiment  determines  the  minimum  contrast  necessary  for  the  identification  of  Snellen  letters. 

1 7.2  Methods 

A  35  mm  slide  of  the  standard  Snellen  letter  chart  shown  in  Figure  86  was  projected  full  size  onto  a  white  matte  surface 
and  viewed  from  the  standard  six  meters  by  five  subjects  with  normal  or  corrected  vision  of  6/6  or  better.  The  contrast 
of  the  chart  was  controlled  by  the  crossed  polaroid  system  discussed  in  Section  A-2.4.  The  mean  luminance  was  20 
foot-lamberts.  In  order  to  insure  that  each  Snellen  line  was  viewed  under  identical  conditions,  the  slide  that  projected 
the  Snellen  chart  was  positioned  to  place  each  line  of  letters  between  two  fixed  blue  markers  before  each  trial.  The 
subjects  were  told  that  they  would  see  a  standard  Snellen  letter  chart — lines  of  capital  block  letters. 

The  threshold  contrast  for  the  detection  of  letters  of  each  Snellen  line  was  used  as  a  reference  level  for  the  contrast 
required  for  letter  identification.  1'he  subject  was  told  to  report  any  in  homogeneity  between  the  blue  markers  as  the 
contrast  of  the  letters  was  slowly  increased  from  zero  contrast  by  the  experimenter.  That  response  determined  the 
contrast  noted  for  detection.  The  contrast  was  slowly  increased  further  until  the  subject,  making  a  forced  guess,  could 
correctly  identify  at  least  5(1  percent  of  the  letters  on  each  line.  This  response  indicated  the  minimum  contrast  necessary 
for  the  identification  of  the  letters  of  the  Snellen  line.  This  method  was  repeated  for  each  subject  for  each  line  from 
6/M)  to  6/5. 

These  methods  provide  only  general  values  of  the  minimum  contrast  for  the  detection  and  identification  of  Snellen 
letters.  Each  different  letter  has  its  own  detection  threshold  which  may  be  slightly  different  from  other  letters  of  the 
same  size  due  to  physical  differences  between  each  letter.  Further,  the  contrast  required  for  at  least  5()  •  .  i  cent  correct 
identification  of  letters  of  each  Snellen  line  does  not  represent  the  same  level  of  subject  performance  because  of  the 
different  number  of  letters  in  each  line.  A  higher  level  of  performance  is  required  to  correctly  report  4  of  8  letters  for 
small  letters  than  to  correctly  report  1  of  2  letters  for  the  large  letters.  However,  since  these  factors  are  present  in  the 
normal  measure  of  Snellen  acuity .  they  are  merely  noted  and  not  corrected  for  in  these  measurements.  These  factors 
could  be  corrected  by  using  binominal  probabilities. 

A  criterion  of  at  least  5(1  percent  correct  identification  of  the  letters  on  each  Snellen  line  was  used  in  an  attempt  to  get  an 
average  contrast  value  of  the  different  letters  on  each  line.  Pilot  studies  showed  that  most  subjects  required  significantly 
more  contrast  to  identify  the  remaining  1  or  2  letters  on  the  smaller  Snellen  lines.  It  was  felt  that  increasing  the  contrast 
to  allow  the  correct  identification  of  all  the  letters  would  not  reflect  the  average  contrast  necessary  for  the  identification 
of  most  letters.  Since  identification  of  over  5(1  [a-rcent  of  the  letters  was  used  as  a  criterion  to  determine  the  minimum 
contrast  of  the  identification  of  each  line.  2.5  <•/ 1  of  each  letter  cannot  bo  assumed  to  be  usid  for  identification.  The 
average  number  of  2  c/ 1  found  front  filtering  of  the  L  11.5  «•/ 1 )  and  E  (2.5  e/ 1 1  is  used  in  the  subsequent  analy  sis  for  the 
minimum  number  of  spatial  frequencies  required  for  the  identification  of  Snellen  letters. 

I  7.3  Results  and  Discussion 

The  mean  contrast  required  for  the  detection  and  identification  of  at  least  .5(1  |M>rcent  of  the  letters  for  each  Snellen  line 
for  the  five  subjects  is  shown  in  Figures  87  and  88.  The  reciprocal  ol  those  contrast  values,  contrast  sensitiv  ity  .  is  shown 
in  Figure  8'>.a.  The  values  of  size  and  spatial  frequency  noted  along  the  abscissa  correspond  to  the  w  idth  and 
fundamental  cycle  of  the  Snellen  letters  for  each  corresponding  Snellen  line. 


The  results,  shown  in  Figures  88  and  86, a  are  as  expected:  higher  contrast  (lower  contrast  sensitivity  I  is  required  for 
both  detection  and  identification  as  the  letters  get  smaller.  Note  that  the  detection  threshold  is  constant  for  Snellen  lines 
6/60  to  6/18.  This  result  appears  reasonable  if  detection  is  based  upon  the  half  and  first  cycles  per  letter  reaching 
threshold,  which,  on  a  line  6/60  to  6/ 18,  range  from  0.6  to  4  c/d.  Since  those  spatial  frequencies  occur  around  the  |>eak 
of  the  contrast  sensitivity  function.  1  to  2  c/d,  they  reach  threshold  with  similar  amounts  of  contrast. 

A  similar  argument  can  be  made  for  the  nearly  simultaneous  detection  and  identification  seen  on  lines  6/60  and  6/36. 
Since  the  relevant  range  of  spatial  frequencies  for  the  detection  and  identification  of  those  letters  can  be  seen,  from 
Figure  84,  to  occur  from  0.6  to  4  c/d  and  those  spatial  frequencies  occur  around  the  peak  of  the  contrast  sensitivity, 
letters  of  those  sizes  will  reach  detection  and  identification  threshold  at  similar  levels  of  contrast. 

It  should  be  noted  that  letters  larger  than  those  on  line  6/60  would  be  expected  to  have  thresholds  similar  to  letters  on 
lines  (>/ Ml  to  6/ 18.  Since  the  letters  are  made  of  bars  and  the  contrast  sensitivity  to  square  waves  is  approximately 
constant  below  about  1  c/d,  the  detection  of  spatial  frequencies  below  1  c/d  would  remain  relatively  constant. 

However,  further  research  will  be  needed  to  determine  how  well  this  approach  will  predict  the  identification  of  objects 
having  fundamental  frequencies  less  than  1  c/d. 

The  curves  in  Figure  80, a  show  that  as  the  letters  get  smaller,  increased  contrast  is  required  to  identify  the  letters  after 
the)  are  detected.  This  result  is  understandable  if  detection  and  identification  of  the  Snellen  letters  on  the  average 
require  the  first  and  second  cycle  per  letter  to  reach  detection  and  identification  threshold,  respectively.  Figure  8i 
shows  that  I  to  2  c/ 1  occurs  at  1 .2  and  2.8  c/d  for  letters  on  line  6/60.  and  12  and  24  c/d  for  letters  on  line  6/6.  Since, 
after  peak  sensitivity,  the  contrast  sensitivity  function  decreases  exponentially  with  increasing  spatial  frequency 
(Campbell  and  Green.  I%3I.  more  contrast  should  be  required  to  identify  letters  on  line  6/6.  where  the  relevant  spatial 
frequencies  are  12  and  24  c/d.  than  when  the  letters  are  on  line  6/60. 

The  minimum  number  of  cycles  of  the  Snellen  letters  that  were  list'd  by  the  subjects  for  detection  and  identification  can 
be  obtained  from  Figure  80,b  in  which  contrast  sensitivity  of  the  detection  and  identification  of  Snellen  letter  is  plotted 
against  linear  spatial  frequency.  Regression  lines  were  determined  using  the  least  squares  method.  The  correlation 
coefficient  (rl  for  the  regression  line  of  the  identification  data  is 0.904.  Two  different  regression  lines  have  been  plotted 
for  the  detection  data.  Note  that  the  first  four  data  points  are  horizontal  because  detection  occurred  at  the  peak  of  the 
contrast  sensitivity  for  those  Snellen  letters.  Since  the  contrast  sensitivity  peak  can  shift  to  somewhat  different  spatial 
frequencies  due  to  experimental  conditions  (see  Section  .3.2. 1 ).  an  argument  can  be  made  that  the  first  four  data  points 
for  detection  threshold  could  change  under  different  viewing  conditions.  Thus,  regression  lines  were  computed  with 
(solid  lines,  r  =  0.6761  and  without  (dashed  line,  r  =  0.6621  the  first  three  data  points.  The  bandwidth  of  the 
identifieation-to-deteelion  threshold  ratio  determined  from  the  intercept  of  the  regression  lines  to  the  spatial  frequency 
axis  is  36/23,  or  2.33  cycles,  using  all  the  detection  data  points  and  32.6/23,  or  2.37  cycles,  excluding  the  first  three 
detection  data  (mints.  These  values  are  in  quite  good  agreement  with  both  the  theor.  tical  analysis  (Section  10.21  and  the 
filtering  analysis  (Section  16.31  that  suggested  that  the  detection  and  identification  of  Snellen  letters  require  about  1.3 
to  2.3  c  / 1 . 

Note  that  this  result  does  not  mean  that  a  (I  to  2.3  c/I  bandwidth  is  being  used  for  detection  and  identification.  Rather, 
it  means  that  some  range  of  spatial  frequencies  determined  by  multiply  ing  the  lowest  spatial  frequency  by  2.3  is  lining 
used.  For  example,  if  detection  is  due  to  a  half-cycle  of  a  letter  reaching  threshold,  then  identification  would  occur 
when  the  2.3  x  0.3  cycle,  or  1 .7.3  <■/  I ,  reaches  threshold. 

The  ratio  of  the  contrast  required  for  identification  versus  detection  of  Snellen  letters  is  also  shown  in  Figure  60.  \ 
regression  line  determined  using  the  least  squares  method  is  shown  to  fit  well  w  ith  a  correlation  coefficient  of  0.’>64. 

Note  that  the  regression  line  crosses  the  spatial  frequency  axis  at  about  30  c/d.  The  second  harmonic  required  for 
identification  would  he  at  00  c.'d.  which  is  the  theoretical  cutoff  spatial  frequency  ,  i.e..  the  limit  of  \  isual  acuity  .  This 
result  supports  the  data  very  well. 

W  h\  should  the  ratio  of  detection  to  identification  of  Snellen  letters  be  a  straight  line?  As  prex  iously  noted,  the  contrast 
sensitiv  ity  function  decreases  exponentially  from  peak  to  minimum  sensitiv ity  w  ith  increasing  spatial  frequency  w lien 
plotted  on  a  log  contrast  sensitivity  versus  lo<»  spatial  frequency  scale.  If  the  ratio  of  spatial  frequencies  used  for  the 
detection  and  identification  of  Snellen  letters  remains  the  same  for  ail  different  size  letters  and  the  detection  and 


identification  thresholds  are  a  function  of  the  shape  of  the  contrast  sensitivity  function,  then  the  ratio  should  be  a 
straight  line  when  plotted  on  a  linear  ratio  versus  log  spatial  frequency  scale. 

To  determine  the  generality  of  the  finding  that  detection  and  identification  of  Snellen  letters  requires  about  2  c/ 1 ,  ratios 
are  plotted  of  contrast  sensitivity  of  1.5,  2,  and  3  spatial  frequencies  from  values  obtained  from  contrast  sensitivity 
measurements  of  the  author  (see  Figures  90  and  91 1.  Also  plotted  is  the  ratio  of  the  threshold  contrast  of  two  spatial 
frequencies  from  Figure  3  of  Campbell  and  Robson  ( 1968).  Slopes  of  the  ratios  of  contrast  sensitivity  curves  are  slightly 
steeper  than  the  slope  of  the  data.  However,  considering  the  different  experimental  conditions  used  to  collect  these  data, 
the  general  agreement  between  the  theoretical  slopes  in  the  data  are  quite  good,  especially  for  the  ratio  of  two  spatial 
frequencies. 

It  is  concluded  that  there  is  good  agreement  between  the  theoretical  and  experimentally  determined  number  of  cycles 
sufficient  for  the  identification  of  Snellen  letters.  Now  that  the  contrast  sensitivity  and  the  general  range  of  spatial 
frequency  required  for  identification  of  Snellen  letters  have  been  determined,  the  Snellen  acuity  of  abnormal  visual 
systems  will  be  determined  from  values  of  the  contrast  sensitivity  functions  obtained  from  certain  abnormal  visual 
systems  to  test  the  predictive  power  of  this  analysis. 
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CHAPTER  18 


THE  RELATIONSHIP  BETWEEN  CONTRAST  SENSITIVITY 
AND  SNELLEN  LETTER  ACUITY  IN  NORMAL  AND  ABNORMAL  VISION 


18.1  Introduction 

The  contrast  sensitivity  of  ten  subjects  with  abnormal  vision  was  measured — one  subject  with  abnormal  retinal 
correspondence,  three  amblvopes.  and  six  patients  having  multiple  sclerosis  (see  Chapters  A- l-A-5).  The  previous 
spatial  filtering  showed  that  1 .5  to  2.5  c/ 1  are  required  for  the  identification  of  Snellen  letters.  The  experiments  in 
Chapter  1 7  determined  the  minimum  contrast  required  for  the  identification  of  at  least  50  percent  of  the  letters  on  each 
line  of  a  standard  Snellen  chart.  These  data,  combined  in  Figure  02,  were  used  in  an  attempt  to  predict  Snellen  acuity 
of  these  subjects  based  upon  their  contrast  sensitivity  measurements. 

18.2  Predicting  Snellen  Acuity  of  Subjects  Having  Amblyopia,  Abnormal  Retinal  Correspondence, 
and  Multiple  Sclerosis 

The  measured  and  predicted  Snellen  letter  acuity  of  ten  subjects  is  shown  in  Figures  03  and  04.  With  the  exception  of 
the  initial  measurements  of  Snellen  acuity  in  subject  A.  H.,  the  Snellen  acuity  was  also  measured  by  other  clinicians. 
The  values  under  the  measured  contrast  sensitivity  column  in  Figure  03  were  obtained  from  contrast  sensitivity 
functions  of  the  subjects  whose  data  are  shown  in  Chapters  A-3  —  A-4.  These  values  represent  the  contrast  sensitivity 
at  the  spatial  frequency  of  2  c/ 1  for  the  Snellen  line  that  determined  the  Snellen  acuity.  Extrapolation  by  eye  was  used 
where  necessary  to  obtain  the  value  of  contrast  sensitivity  below  those  measures.  Since  the  Snellen  letters  are  made  from 
bars,  each  value  of  contrast  sensitivity  was  multiplied  by  1 .27  to  correct  for  the  difference  in  detection  between  the 
fundamental  frequency  of  square-  and  sine-wave  gratings.  The  predicted  Snellen  acuity  was  determined  from  the 
contrast  sensitivity  values  listed  in  Figure  ‘12. 

The  contrast  required  to  identify  at  least  50  percent  of  the  Snellen  letters  on  each  Snellen  line  and  the  average  second 
spatial  frequency  of  the  letters  required  for  identification  provided  a  pair  of  numbers  that  represent  one  point  on  the 
contrast  sensitivity  function.  V  alues  of  contrast  sensitivity  close  to  or  exceeding  the  predicted  contrast  sensitiv  ity 
determined  the  predicted  Snellen  letter  acuity.  For  example,  the  predicted  contrast  sensitivity  for  a  measured  Snellen 
letter  acuity  of  <>/(>  is  7. 1  at  24  c/d.  If  the  measured  contrast  sensitivity  at  24  c/d  was  7. 1  or  more,  then  the  predicted 
Snellen  acuity  was  also  (>/(•.  If  the  measured  contrast  sensitivity  was  less  than  7. 1 .  the  pair  of  spatial  frequencies  and 
contrast  sensitivities  for  the  next  largest  Snellen  lines  were  compared  until  the  measured  values  were  in  agreement  with 
the  predicted  values.  The  values  in  the  two  columns  of  the  predicted  Snellen  letter  acuity  were  determined  from  both 
the  average  and  the  ±  2  standard  error  range  of  contrast  sensitivity  required  for  the  detection  of  the  Snellen  letters. 

A  comparison  between  the  predicted  and  measured  Snellen  letter  acuity  shows  perfect  agreement  for  8  of  22  eyes,  and 
lb  of  22  eyes  can  be  predicted  within  one  Snellen  line  using  the  average  contrast  sensitivity  criteria.  I  The  two  different 
sets  of  Snellen  acuity  of  subject  A. II.  are  considered  separately  .  I  Using  the  ±  2  standard  error  criteria,  the  Snellen 
letter  acuity  predicted  using  contrast  sensitivity  shows  1 1  of  22  eyes  in  agreement,  and  1 7  of  22  eyes  can  be  predicted 
within  one  Snellen  line.  The  Snellen  letter  acuity  of  the  other  5  of  22  eyes  can  be  predicted  within  two  Snellen  lines. 

There  is  better  agreement  between  the  measured  and  the  predicted  Snellen  letter  acuity  using  the -2  standard  error  than 
the  average  standard  error  of  the  contrast  sensitivity  required  for  the  identification  of  the  Snellen  letters.  These  results 
suggest  that  contrast  sensitiv  ity  used  to  predict  Snellen  acuity  is  somewhat  higher  than  that  used  by  the  subjects  for 
their  measured  Snellen  acuity.  ( >ne  obvious  reason  for  this  result  is  that  the  predicted  Snellen  acuity  was  based  on  less 
than  1(H)  (lercent  letter  identification.  The  measured  Snellen  acuity  typically  required  the  identification  of  all  the 
Snellen  letters  on  the  line  that  determined  acuity  .  Thus,  requiring  all  the  letters  on  a  line  to  lx>  identified  would  require 
more  contrast  than  was  used  in  these  experiments.  More  contrast  would  reduce  contrast  sensitivity  and  increase  the 
agreement  between  the  measured  and  the  predicted  Snellen  letter  acuity  based  iqx>n  the  average  contrast  sensitiv  ity 
required  for  the  identification  of  the  Snellen  letters.  ( >f  course,  other  factors,  such  as  the  use  of  different  criteria  and 
task  for  the  detection  of  gratings  and  the  identification  of  Snellen  letters  and  different  ex|>erimental  conditions,  would 
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have  an  effect  and  should  be  investigated. 


It  is  concluded  that  there  is  good  agreement  between  predicted  and  measured  Snellen  letter  acuity  based  upon  contrast 
sensitiv  ity  measurements.  These  results  are  especially  encouraging  because  of  the  different  patterns  being  compared, 
i.e..  one-dimensional  sine-wave  gratings  and  two-dimensional  letters.  Furthermore,  there  was  a  difference  in 
equipment,  methods,  and  criteria  used  to  obtain  the  data.  Such  good  agreement  between  predicted  and  measured 
Snellen  letter  acuity  suggests  the  power  of  contrast  sensitivity  measurements  for  accurate  descriptions  of  the  overall 
filtering  characteristics  of  the  visual  system  used  for  the  detections  and  identification  of  objects. 

IK. 3  Relating  Snellen  Letter  Acuity  to  the  Contrast  Sensitivity  of  an  Amblyope  with  a  Notch  in  Her  Contrast 
Sensitivity  Function 

From  Figure  92,  the  initial  Snellen  acuity  of  6/36  obtained  from  amblyope  A.  H.  requires  up  to  4  c/d  of  resolution  for 
the  first  2  c/ 1 .  According  to  her  initial  contrast  sensitivity,  as  shown  in  Figure  99,  this  range  of  spatial  frequencies 
corresponds  to  the  region  of  decreasing  sensitivity  after  the  peak  sensitivity  at  1  c/d.  Although  the  4  c/d  component  of 
the  letter  falls  within  the  least  sensitive  region  of  the  notch,  the  contrast  sensitivity  at  4  c/d  is  similar  to  that  at  10  c/d 
and  greater  than  that  of  5  c/d.  The  data  in  Figure  99  show  that  the  Snellen  acuity  of  6/36  of  the  amblyopic  eye  of  A.  H. 
is  a  factor  of  T  less  than  the  acuity  of  her  normal  eye.  6/5.  The  spatial  frequency  for  2  c/1  of  Snellen  line  6/5  is  28.8  c/d 
t  Figure  841.  If  the  acuity  in  her  amblyopic  eye  is  a  factor  of  7  less  than  that  of  the  normal  eye.  then  the  highest  spatial 
frequency  that  was  used  by  her  amblyopic  eye  for  Snellen  acuity  is  about  4  c/d.  Since  that  spatial  frequency 
corresponds  to  2  c/1  for  letters  on  line  6/36,  those  values  of  spatial  frequency  greater  than  about  4  c/d  —  after  the  least 
sensitive  point  in  her  notch  —were  not  used  for  the  identification  of  Snellen  letters.  Indeed,  from  Figures  92  and  99  it  can 
be  seen  that  the  sensitivity  at  those  spatial  frequencies  is  too  low  to  provide  spatial  information  for  Snellen  acuity 
greater  than  about  6/36.  Thus,  the  factor  of  7  decrease  in  Snellen  acuity  from  her  normal  to  her  abnormal  eye 
corresponds  well  to  the  factor  of  7  decrease  of  the  usable  range  of  spatial  frequencies  as  determined  from  the  contrast 
sensitivity  in  Figure  92. 

A  similar  analysis  can  be  made  for  the  increased  contrast  sensitivity  from  2  to  8  c/d  and  the  concomitant  increase  of 
Snellen  acuity  from  6/36  to  6/18  in  her  amblyopic  eye.  From  Figure  84.  the  first  2  c/1  on  line  6/18  occurs  at  8  c/d.  The 
region  of  greatest  increase  in  contrast  sensitivity  in  her  amblyopic  eye  is  from  2  to  9  c/d  (Figure  1 10).  The  sensitivity  at 
4  c/d  increased  by  about  a  factor  of  10  from  her  initial  measurements.  The  increased  contrast  sensitivity  from  2  to  9  c/d 
is  in  good  agreement  with  the  predicted  sensitivity  required  to  identify  the  Snellen  letters  on  line  6/18. 

These  results  provide  additional  evidence  that  suggests  the  existence  of  narrowly  tuned  mechanisms  in  the  visual 
system  —  channels—  which  are  used  in  the  identification  of  complex  objects. 

The  initial  Snellen  acuity  of  amblyope  A.  H.  was  6/5  (Rl  and  6/36  (LI.  Her  initial  contrast  sensitivity  predicted  6/9  lH  I 
and  6/60  I  LI.  Although  this  analysis  is  close  to  predicting  her  initial  Snellen  acuity,  further  analyses  of  these  results  are 
interesting.  The  two  letters  A  and  V  from  the  Snellen  chart  of  Figure  86.  required  for  identification  for  6/36  acuity, 
were  not  filtered  to  determine  the  minimum  number  of  spatial  frequencies  necessary  for  identification.  The  general 
criterion  of2c/l  for  letter  identification  may  be  too  high.  A  criterion  of  l.5c/l  does  predict  6/5  (R I  and  6/36  IL).  One 
final  note  is  that  her  initial  contrast  sensitivities  are  among  the  lowest  obtained.  Slightly  higher  values  would  have 
resulted  in  agreement  between  measured  and  predicted  Snellen  acuity  . 

18.4  Predicting  Snellen  Acuity  from  Data  of  Other  Amblyopes 

The  number  of  cycles  required  for  the  identification  of  Snellen  letters  versus  contrast  sensitivity,  shown  in  Figure  92. 
ap[>ears  to  enable  prediction  of  Snellen  letter  acuity  from  contrast  sensitivity  measurements  obtained  by  other 
researchers.  Levi  and  Harwerth  ( 19771  showed  the  contrast  sensitivity  measurements  of  three  subjects.  Subjects  M.  M., 
L.  N..  and  M.  R.  had  Snellen  acuities  of  20/20  (K-6/61.  20/2001 1. -6/601;  20/200  (R-0/601.  20/20(1,-6/601;  and  20/30 
(R-6/91.  20/20  (L*0/6I.  The  Snellen  letter  acuities  of  20/20  were  just  outside  the -2  standard  error  of  the  predicted 
contrast  sensitiv  ity  but  were  closer  to  20/20  than  20/30.  the  next  Snellen  line.  The  20/200  and  20/50  acuities  could  be 
predicted  from  Figure  92.  Considering  the  different  Snellen  test  charts,  equipment,  methods,  and  criterion  differences 
in  the  measurements,  the  agreement  between  their  data  and  the  results  reported  here  sup[>orts  the  predictive  |w>wer  of 
the  previous  analysis. 


18.5  Summary 


The  abnormal  filtering  characteristics  of  individuals  with  amblyopia  and  multiple  sclerosis  were  determined.  One 
amblyope  had  a  unique,  highly  spatial  frequency  selective  region  (4  to  8  c/d  I  of  decreased  contrast  sensitivity  (notch  I  to 
both  stationary  and  moving  sine-wave  gratings.  Increased  sensitivity  of  the  spatial  frequencies  of  that  highly  selective 
notch  occurred  with  a  concomitant  increase  in  two  lines  of  Snellen  acuity. 

An  attempt  was  made  to  link  the  results  of  filtering  objects  reported  in  Part  III  of  this  dissertation  with  the  Snellen 
letter  acuity  of  these  subjects.  Full  predictive  power  required  knowledge  of  the  minimum  contrast  required  for  the 
identification  of  Snellen  letters.  An  experiment  provided  these  data.  The  minimum  contrast  and  range  of  spatial 
frequencies  required  for  Snellen  letter  acuity  were  related  to  the  two  cycles  per  letter  generally  found  necessary  to 
provide  the  minimum  amount  of  information  required  to  identify  Snellen  letters.  That  data  enabled  Snellen  letter 
acuity  to  be  predicted  within  one  Snellen  line  for  17  of  22  eyes. 

In  sum.  the  following  approach  has  been  shown  to  successfully  predict,  at  least  to  a  first  approximation,  the 
identifications  of  complex  objects  —  Snellen  letters  —  from  the  spatial  filtering  characteristics  of  the  visual  system  using 
contrast  sensitivity  measurements.  First,  the  minimum  number  of  spatial  frequencies  sufficient  for  identification  w  as 
determined  by  spatial  filtering.  Second,  those  spatial  frequencies  sufficient  for  identification  were  required  to  exceed 
contrast  threshold  determined  from  contrast  sensitivity  measurements  of  the  observer.  This  approach,  similar  to  that 
used  by  Campbell  and  Robson  ( 1%8)  to  determine  the  detection  and  identification  of  square  waves,  assumes  first-order 
approximation  of  linearity  of  the  visual  system.  The  extent  to  which  this  approach  will  predict  the  identification  of  the 
other  objects  will  require  further  research  (discussed  further  in  Section  19.101.  However,  these  results  clearlv  support 
the  findings  in  the  first  part  of  the  dissertation  which  suggest  that  a  one-  to  two-octave  channel  can  provide  sufficient 
information  for  the  identification  of  objects. 
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CHAPTER  19 


GENERAL  DISCUSSION  AND  CONCLUSIONS  ABOUT  HOW  THE  FILTER 
CONCEPT  RELATES  TO  THE  OVERALL  PROBLEMS  OF  PERCEPTION 


19.1  Introduction 

The  previous  chapters  have  shown  how  the  concept  of  spatial  filtering,  based  upon  known  filtering  properties  of 
mechanisms  found  in  mammalian  visual  systems,  can  offer  a  parsimonious  explanation  of  many  aspects  of  visual 
perception.  There  are  many  alternatives  to  one  or  more  of  the  explanations  of  the  filtered  images.  It  is  an  impossible 
task  to  counter  all  the  potential  criticisms  of  the  somewhat  cavalier  manner  in  which  certain  long-standing  problems  in 
perception  have  been  treated,  e.g.,  the  Gestalt  principles.  However,  the  aim  of  this  dissertation  is  the  presentation  of  a 
sufficient  number  of  examples  to  demonstrate  the  parsimony  of  the  filter  concept  with  many  different  aspects  of 
perception. 

Several  questions  immediately  arise  about  how  the  results  may  fit  into  the  overall  scheme  of  visual  processing.  What  is 
the  relationship,  for  example,  between  spatial  filtering  and  memory  or  selective  attention?  What  are  the  implications 
for  machine  pattern  recognition?  What  problems  in  visual  perception  does  the  filter  concept  appear  to  solve?  And, 
what  problems  remain?  These  and  other  questions  will  be  considered  in  this  final  chapter. 

19.2  The  Relationship  between  Spatial  Filtering  and  Higher  Visual  Processing 

Although  the  concept  of  spatial  filtering  by  channels  in  the  visual  system  appears  to  offer  reasonable  and  testable 
explanations  of  many  perceptual  problems,  there  is  still  an  overall  cognitive  structure  that  has  to  deal  with  the  filtered 
information.  For  example,  as  one  moves  from  the  simple  multistable  objects  to  more  complex  objects  such  as  the 
man-girl  figure  of  Fisher  ( 1968*,  decision  boundaries  that  are  a  natural  concomitant  to  any  pattern  classification  system 
must  also  be  taken  into  account  (Ginsburg,  et  al.,  1976). 

Although  the  concept  of  a  hierarchy  of  filtered  images  appears  to  predict  the  possible  patterns  that  are  available  to  an 
observer,  it  cannot  predict  which  pattern  will  be  seen  and  reported  by  an  observer  at  a  particular  moment  under  free 
viewing  conditions  (excluding  biasing  by  adaptation,  masking,  etc. ).  This  aspect  of  visual  processing  is  suggested  to  be 
one  of  the  major  cognitive  elements  of  perception  in  which  “nonsensory  ”  concepts  of  Hebb  (1949),  such  as  attention, 
context,  motivation,  and  set,  are  relevant.  Indeed,  based  on  data  from  several  perceptual  tasks,  it  has  been  argued  that 
the  filtering  concept  must  be  used  in  conjunction  with  such  concepts  as  “set”  and  “selective  attention,”  even  for 
machine  pattern  recognition  (Ginsburg,  1971, a, b;  Ginsburg,  1973;  Ginsburg,  et  al.,  1976). 

Perception  involves  both  physiological  and  cognitive  mechanisms.  A  major  problem  is  the  isolation  of  the  different 
aspects  of  perception  that  are  predominantly  due  to  physiological  and/ or  cognitive  mechanisms.  The  filtering  approach 
appears  to  offer  a  powerful  tool  in  the  solution  of  such  problems.  Clearly,  cognitive  mechanisms  must  be  constrained  by 
the  nature,  i.e.,  the  bandwidth,  of  the  physiological  mechanisms. 

19.3  Implications  for  Machine  Pattern  Recognition 

It  was  the  initial  success  of  machine  pattern  recognition  using  filtered  transforms  that  prompted  the  author  to  apply  the 
concept  of  spatial  filtering  to  problems  of  visual  perception  (Ginsburg,  1971, a, b). 

Kabrisky.  Tallman,  Day,  and  Radoy  ( 1970)  reported  the  results  of  successful  machine  pattern  recognition  using 
hand-printed  alphanumeric  patterns  based  on  a  model  of  the  human  visual  system.  The  central  concept  of  the  machine 
algorithm  is  spatial  filtering  in  a  two-dimensional  transform  domain.  The  input  pattern  is  transformed  and  spatially 
filtered.  The  low  frequency  components  of  the  input  pattern  are  then  compared  to  the  low  frequency  components  of 
certain  patterns  stored  in  memory  using  a  Euclidean  distance  metric  (cross-correlation ).  Minimum  Euclidean  distance 
between  the  input  pattern  and  a  pattern  stored  in  memory  classifies  them  as  the  same. 
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It  is  a  general  tenet  of  pattern  recognition  that  if  the  correct  features  of  patterns  are  chosen,  then  the  measure  needed  to 
classify  patterns  as  the  same  will  be  simple.  Incorrect  feature  selection  usually  leads  to  complex  classification  schemes. 
Cross-correlation  of  patterns  (minimum  Euclidean  distance!  is  a  simple  classification  measure. 

A  review  of  the  early  results  of  application  of  the  filtered  transforms  to  the  general  classification  problem  can  be  found 
in  Carl  and  Hall  (1972).  It  is  interesting  to  note  that,  in  general,  the  three  or  four  harmonics  that  were  required  for  a 
high  percentage  of  correct  classifications  of  a  large  number  of  different  objects  are  the  same  as  those  found  to  be  so 
important  for  most  of  the  filter  demonstrations  presented  in  this  dissertation. 

Machine  pattern  recognition  based  upon  filtered  transforms  is  not  perfect.  It  fails  in  some  cases,  but  generally  it  fails 
with  the  same  patterns  that  a  human  observer  also  fails  to  classify.  The  machine,  so  far,  is  generally  limited  to  isolated 
objects.  It  is  hoped  that  some  of  the  rules  for  the  segregation  of  objects  presented  in  this  dissertation  will  help  solve  the 
segregation  problem. 

The  point  made  in  this  section  is  that  the  filter  concept  that  has  been  presented  in  this  dissertation  does  work  for 
machine  pattern  recognition  for  the  class  of  objects  tested  on  the  machine,  e.g.,  hand-printed  alphanumerics. 

Extension  of  some  of  the  results  of  the  filtering  presented  in  this  dissertation  may  help  in  the  construction  of  a  “seeing” 
machine.  Minimally,  the  filter  concept  based  on  visual  information  processing  should  help  reduce  the  choice  between 
numerous  algorithms  which  exist  for  feature  detector  concepts. 

The  author  has  attempted  to  combine  the  previous  machine  pattern  recognition  techniques  with  the  same  results  of  the 
spatial  filtering  reported  here  into  schemes  for  machine  pattern  recognition  (Ginsburg,  19731. 

19.4  Some  Problems  in  Object  Recognition  That  the  Spatial  Filter  Concept  Can  Help  to  Solve 

Sutherland  ( 1968)  listed  twelve  sets  of  facts  which  he  claimed  should  be  explained  by  any  theory  of  object  recognition. 
Since  that  list  covers  the  major  problems  in  object  recognition,  it  is  used  here  as  a  general  guide  in  discussing  what  the 
filtering  concept  can  and  cannot  offer  in  the  solution  of  such  problems. 

19.4.1  Size  Invariance 

Biological  visual  systems  are  able  to  identify  similar  objects  even  when  their  size  varies  over  a  considerable  range.  If  an 
object  is  decomposed  into  components  of  size  or  spatial  frequency  (as  by  Fourier  analysis),  then  changing  the  size  of  an 
object  will  change  the  range  of  size  or  spatial  frequency  that  the  object  is  decomposed  into  by  a  scaling  factor  (as  shown 
in  Figure  3).  The  ratio  of  the  sizes  or  spatial  frequencies  will  remain  the  same.  Thus,  if  a  system  is  able  to  extract  the 
ratio  of  sizes  or  spatial  frequencies  of  an  object,  then  classification  of  different  sizes  of  objects  could  be  easily  achieved. 
Blakemore  and  Campbell  ( 1969)  recognized  the  importance  of  such  a  scheme  to  account  for  size  invariance. 

The  simplicity  and  power  of  such  a  scheme  is  even  more  attractive  when  the  size  or  spatial  frequency  components  are 
based  upon  the  overall  size  of  the  object.  It  was  shown  previously  that  three  to  four  spatial  frequencies  easily  provide 
the  basic  information  about  the  form  of  an  object.  Thus,  the  ratio  of  those  few  spatial  frequencies  could  be  used  to 
provide  the  information  required  to  classify  an  object  over  large  ranges  of  different  sizes,  assuming  that  the  relative 
amplitudes  remain  constant. 

19.4.2  Invariance  to  Retinal  Position 

Objects  can  be  correctly  classified  as  the  same  over  different  regions  of  the  retina.  This  ability  is  limited  by  the  loss  of 
visual  acuity  in  the  peripheral  retina.  The  power  of  the  low  spatial  frequencies  to  provide  uniform  form  information 
across  the  inhomogeneous  retina  was  discussed  in  Section  5.2.1.  Another  attractive  property  of  the  Fourier  transform  is 
that  the  magnitude  of  the  spatial  frequencies  is  translationally  invariant,  e.g.,  see  Bracewell  (1965).  If  phase  is 
eliminated,  there  are  very  many  different  patterns  that  could  be  reconstructed  from  spatial  frequencies  that  have  the 
same  magnitude.  However,  consider  two  different  kinds  of  phase  —  explicit  and  implicit.  Explicit  phase  will  be  used  to 
define  the  relative  position  of  the  object  with  respect  to  retinal  space.  Implicit  phase  will  be  used  to  define  the  relative 
position  of  the  spatial  frequencies  of  the  object  with  respect  to  the  object.  In  other  words,  certain  mechanisms  in  the 
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visual  system  that  respond  to  stimulation  at  specific  regions  of  visual  space  provide  explicit  phase  information  about 
that  object.  Certain  mechanisms  that  respond  to  certain  spatial  information  and  are  independent  of  where  the  object  is 
in  visual  space  provide  implicit  phase  along  with  spatial  information  of  the  object.  Pollen,  Lee,  and  Taylor  •  1971 1 
recognized  that  complex  cells  have  the  property  of  translational  invariance.  The  role  of  the  complex  cells  may  be  to 
extract  position  invariant  spatial  information  about  objects.  If  the  magnitude  ratio  of  spatial  frequencies  of  an  object  is 
extracted  along  with  an  implicit  phase,  then  size  and  position  invariant  object  recognition  may  be  accomplished. 

19.4.3  Recognition  of  Objects  When  Their  Contrast  Is  Reversed 

The  decrease  in  the  recognition  of  objects  when  their  contrast  is  reversed  was  discussed  in  Section  10.7.  The  harmonic 
analysis  of  an  object,  typified  by  Fourier  analysis,  will  be  somewhat  different  for  objects  whose  contrast  is  reversed, 
suggesting  one  reason  why  photographic  negatives  are  difficult  to  recognize. 

19.4.4  Equivalence  of  Objects  That  Are  Solid  or  Outline 

Data  in  Section  10.6  showed  that  information  from  the  low  spatial  frequencies  of  an  edge  type  portrait  compared  well  to 
similar  information  of  other  portraits.  In  Section  12.7  the  subjective  contours  were  shown  to  be  physically  present  from 
the  low  spatial  frequencies  of  a  solid  and  outline  Kanizsa  triangle. 

It  is  not  clear  whether  only  one  global  band  of  low  spatial  frequencies  could  be  used  for  the  recognition  of  all  line  objects. 
It  may  be  that  segmentation  of  parts  of  such  objects  must  come  first  to  isolate  regions  over  which  the  low  spatial 
frequencies  are  then  used  for  recognition,  as  was  discussed  in  Section  13.3.  Much  more  work  is  needed  in  this  area 
before  a  clear  picture  will  form. 

19.4.5  (Vonequivalence  of  Recognizing  Rotated  Shapes 

As  Sutherland  1 19681  pointed  out,  a  major  mistake  that  many  researchers  in  the  field  of  pattern  recognition  make  is  to 
assume  that  a  rotated  object  will  be  treated  as  the  original  nonrotated  object. 

Several  examples  in  the  preceding  sections,  e.g.,  Figures  19  and  53,  showed  how  the  shape  of  objects  will  change  as  a 
function  of  the  anisotropic  spatial  filtering  properties  of  the  visual  system. 

Machine  pattern  recognition  studies  based  upon  filtered  two-dimensional  transforms  (see  Section  19.3)  failed  to  classify 
objects  correctly  when  they  were  rotated  more  than  about  ±  1 5  degrees.  Of  course,  this  angle  depended  on  the  shape  of 
the  object  as  well  as  the  bandwidth  of  the  filter  that  was  used. 

The  inability  of  the  machine  to  classify  rotated  objects  correctly  corresponds  to  human  inability,  in  general,  to  classify 
rotated  objects.  Good  agreement  between  the  machine  and  human  observers  was  obtained  by  correlating  the  errors 
made  by  the  machine  with  the  errors  made  by  subjects  in  the  identification  of  rotated  capita)  letters  (Ginsburg. 

1971  .a, b).  However,  this  work  is  only  a  beginning.  Clearly,  there  are  certain  objects,  such  as  letters  and  faces,  that  are 
quite  difficult  to  recognize  when  they  are  rotated.  There  are  many  other  objects  that  present  no  problem,  e.g., 
furniture  and  cutlery.  Whether  several  different  orientations  of  such  objects  are  stored  in  memory  because  they  are  seen 
often  having  different  orientations  is  not  clear.  It  is  noted  that  typesetters,  because  they  are  exposed  to  various 
orientations  of  printed  material,  can  recognize  such  material  with  ease  at  any  orientation.  This  observation  suggests 
that  the  facility  to  recognize  rotated  objects  is  at  least  partly  a  learned  ability. 

Thus,  if  the  visual  system  encodes  spatial  information  about  objects  by  using  orientation  channels,  then  the  price  that  is 
paid  is  the  loss  of  rapid  object  recognition  of  rotated  objects  (Blakemore  and  Campbell,  1969).  The  two-dimensional 
transform  analysis  presented  in  this  dissertation  offers  a  quantitative  means  with  which  to  investigate  the  recognition  of 
rotated  objects. 

19.4.6  Similarity  and  Confusion  Between  Shapes 

The  successes  as  well  as  the  failures  of  the  machine  in  recognizing  objects  prompted  many  experiments  of  similarity 


94 


A 


J.  vjwW  oma*  *■>*  •  -> 


judgement  with  a  wide  variety  of  objects  (see  Ginsburg,  et  al.,  1976  for  a  review).  In  those  studies,  subjects  as  well  as  the 
machine  were  required  to  rank  order  the  similarity  of  objects.  The  machine  rank  ordered  the  objects  by 
cross-correlating  (minimum  Euclidean  distance)  only  their  low  spatial  frequencies.  The  correlation  coefficient  between 
the  machine  and  the  subjects  from  such  experiments  was  generally  above  0.9.  The  low  spatial  frequencies  of  objects 
used  to  create  the  filtered  images  such  as  shown  in  this  dissertation  seem  to  capture  at  least  some  of  the  important  form 
information  that  can  be  used  in  classifying  the  basic  form  of  objects  as  the  same  or  different. 

19.4.7  Recognition  with  Local  Variations  of  Shape 

Pattern  recognition  with  local  distortion  of  shape  is  accomplished  quite  easily  as  other  people’s  handwriting  is  read 
(assuming  basic  legibility,  of  course).  Machine  pattern  recognition  has  also  been  quite  successful  with  such  tasks  (see 
Section  19.3).  Examples  have  been  shown  of  discriminably  different  objects  having  similar  forma  when  low-pass 
filtered,  e.g..  Figure  26.  Low-pass  filtering  may  offer  a  quite  reasonable  solution  to  the  local  distortion  problem. 

19.4.8  Segmentation 

The  segmentation  problem  (physically  isolating  objects  or  parts  of  objects  in  complex  scenes  for  further  analysis)  is 
perhaps  one  of  the  major  remaining  problems  to  be  understood  and  solved.  The  isolation  of  edges  is  clearly  an 
important  prerequisite  for  segmentation,  as  many  previous  researchers  have  realized.  The  plethora  of  edge  extracting 
techniques  exists  mainly  because  of  the  almost  infinite  number  of  features  and  algorithms  that  can  be  specified. 

Constraining  segmentation  techniques  to  those  that  the  visual  system  may  use  greatly  delimits  that  number.  The  author 
has  used  a  simple  first  order  metric,  based  on  contrast  differences  that  appear  to  exist  to  some  extent  at  different  levels 
of  the  visual  system,  as  an  aid  to  segmentation;  but  this  work  has  far  to  go.  The  visual  system  may  be  using  information 
from  a  hierarchy  of  filtered  images  to  aid  segmentation.  It  also  may  be  the  case  that  spatial  frequency  (size), 
orientation,  and  contrast  can  be  used  only  as  general  information  to  aid  segmentation.  Clearly,  there  is  no  evidence  that 
the  visual  system  has  the  capability  to  extract  values  of  those  properties  to  the  degree  of  accuracy  that  has  been 
suggested  by  other  researchers,  e.g.,  Marr  (1976). 

The  inhomogeneous  filtering  properties  of  the  retina  may  play  an  important  part  in  segmentation  in  many  ways.  For 
example,  it  reduces  the  amount  of  detail  available  across  the  retina  for  any  segmentation  process.  By  providing  the 
visual  system  with  an  inhomogeneous  filtered  image,  it  almost  has  to  force  global  processes  onto  the  lower  spatial 
frequencies  of  objects  that  cross  through  the  central  retina.  The  use  of  the  low  spatial  frequencies  of  scenes  should 
reduce  the  amount  of  information  required  to  isolate  large  objects,  as  shown  by  the  filtered  images  in  Section  1 1 .4. 

19.4.9  Recognition  of  Complex  Scenes 

Sutherland  (1968)  noted  that  the  human  can  recognize  a  complex  scene  in  presentation  times  as  brief  as  100 
milliseconds.  Although  small  inconsistencies  may  not  be  detected,  large  systematic  ones  are  easily  noticed.  If.  for 
example,  a  patch  of  some  object  is  presented  in  some  quadrant  of  a  picture  or  a  blank  is  left  in  some  scene,  it  is  easily 
reported.  However,  provided  that  there  are  no  gross  differences,  small  inconsistencies  may  go  undetected.  The  paradox 
is:  If  all  the  information  about  the  scene  is  being  processed,  why  can't  the  details  of  each  part  of  the  picture  be  reported 
(Sutherland,  19681? 

One  way  that  the  preceding  observations  are  a  paradox  is  if  the  assumption  is  made  that  the  visual  system  always 
processes  all  the  information  in  a  complex  scene  all  the  time.  Many  researchers  assume  that  because  all  the  detailed 
information  can  be  described  in  a  complex  scene,  the  visual  system  must  be  processing  all  that  detail  as  the  scene  is 
viewed.  However,  as  the  introduction  to  this  dissertation  and  subsequent  reviews  of  the  filtering  characteristics  of 
mammalian  visual  systems  stressed,  much  information  has  to  be  thrown  away  at  various  levels  in  the  visual  system. 

Since  the  low  spatial  frequencies  of  objects  and  textures  have  been  shown  to  convey  much  information  about  shape, 
symmetry,  etc.,  then  the  details— i.e.,  the  high  spatial  frequencies— need  not  be  processed  at  all  for  form  recognition. 

In  other  words,  there  is  no  paradox  if  the  visual  system  uses  spatial  filters  to  segregate  relevant  from  irrelevant 
information  about  objects  at  very  early  stages  of  processing. 

The  price  that  the  visual  system  pays  for  such  a  scheme  is  susceptibility  to  certain  visual  illusions  and  other  "failures" 


such  as  the  previous  paradox  suggested  by  Sutherland.  However,  this  is  a  small  price  to  pay  (or  rapid,  global 
processing.  It  seems  that  the  evolutionary  pressures  that  shaped  the  design  of  visual  systems  were  survival  needs.  The 
early  visual  systems  had  to  do  processing  with  minimum  information  in  a  rapid  manner  to  determine  whether  the 
organism  was  to  eat  or  be  eaten.  The  organism  could  not  care  if  there  was  any  detailed  information  above  that  which 
was  required  to  do  that  task.  Thus,  spatial  detail  exceeding  that  which  is  necessary  to  perform  specific  tasks  becomes 
redundant. 

As  pointed  out  in  the  previous  sections  on  segmentation,  more  research  will  have  to  be  done  before  the  analysis  of 
complex  scenes  can  be  explained.  The  complex  scenes  that  were  processed  in  this  dissertation,  e.g.,  Figure  44,  are  just  a 
beginning.  However,  it  seems  clear  that  such  an  analysis  should  be  approached  by  considering  the  minimum  amount  of 
information,  from  low-pass  filtered  images,  required  to  perform  certain  tasks  rather  than  high  spatial  frequency  details. 

19.4. 10  Perceptual  Learning 

Sutherland  (1968)  suggested  that  it  is  important  for  an  adequate  theory  of  pattern  recognition  to  be  able  to  account  for 
perceptual  learning,  i.e.,  the  ability  to  recognize  that  a  new  object  belongs  to  a  certain  class  of  objects  that  have  been 
learned. 

Tallman  ( 1970)  used  the  basic  concept  of  a  filtered  transform  to  build  an  adaptive  pattern  recognition  scheme.  Once  the 
machine  was  trained  on  an  initial  set  of  objects  (the  low  spatial  frequencies  of  similar  objects  were  averaged),  any 
subsequent  errors  in  classification  of  new  objects  resulted  in  a  shift  in  the  average  spatial  frequencies  to  include  the  new 
object.  In  other  words,  the  decision  boundary  based  upon  the  feature  space  of  the  low  spatial  frequencies  was  shifted  to 
accept  this  new  pattern.  The  machine,  albeit  in  a  crude  manner,  learned.  It  has  a  long  way  to  go  to  compete  with 
human  performance,  but  it  is  a  powerful  beginning  that  is  consistent  with  the  concepts  reported  in  this  dissertation. 

The  results  of  filtering  suggest  that  classification  of  objects  can  be  based  on  information  from  their  low  spatial  frequencies. 
If  the  visual  system  is  indeed  storing  just  a  few  low  spatial  frequencies  about  objects  in  memory,  the  classification  of  objects 
into  certain  classes  can  be  made  quite  easily.  There  are  clearly  other  higher  perceptual  processes  involved  with  learning 
than  just  the  extraction  of  certain  ranges  of  spatial  frequencies.  However,  the  filtering  reported  here  offers  at  least  a 
quantitative  beginning  with  which  to  look  at  those  other  processes. 

19.4.11  Redundancy 

Sutherland  1 19681  used  Julesz-type  random  dot  pictures  as  an  example  of  the  failure  of  the  visual  system  to  see 
differences  between  certain  pictures  even  though  acuity  limitations  were  not  involved.  He  also  asked  how  “We  .  .  . 
take  advantage  of  the  redundancy  of  our  visual  environment  in  forming  the  classes  of  picture  that  we  do  form."  As 
discussed  in  Section  1 1 .4,  the  filtering  of  random  dot  patterns  can  offer  a  simple  explanation  for  the  first  observation. 

The  discussion  of  machine  classification  in  Section  19.3  and  the  low-pass  filtered  images  shown  in  this  dissertation 
suggest  an  answer  to  the  second  question.  Redundancy  of  information  plays  an  important  role  in  communication 
systems.  The  results  of  the  filtering  of  many  objects  have  shown  that  the  high  spatial  frequencies  are  redundant  for  the 
classification  of  those  objects. 

If  one  point  is  to  be  made  from  the  filter  concept,  it  is  that  spatial  information  required  for  the  classification  of  objects 
and  the  spatial  information  about  an  object  that  is  redundant  are  interrelated.  It  is  perhaps  no  accident  that  the 
low-pass  filtering  provides  the  basic  form  of  objects  with  the  minimum  of  spatial  information.  Consider  the  tremendous 
amount  of  redundancy  in  pattern  information  that  has  been  shown.  A  7  by  7  matrix  of  spectral  coefficients  provided 
sufficient  information  for  the  classification  of  many  objects.  Furthermore,  such  information  was  necessary  for  the 
generalization  of  forms  that  were  made  up  of  discrete  pattern  elements.  A  7  by  7  matrix  provides  %  spectral  coefficients 
if  the  zero  spatial  frequency  term  is  excluded.  However,  half  of  these  coefficients  are  redundant  due  to  the  symmetry 
properties  of  the  sine  and  cosine  parts  of  the  Fourier  transform.  Thus,  24  real  (cosine)  and  24  imaginary  (sine) 
components  have  been  used.  (Both  sine  and  cosine  terms  are  necessary  in  order  to  preserve  the  phase  information.  I 

It  should  be  emphasized  that  this  7  by  7  matrix  is  a  general  value.  Some  objects  will  require  less  information  and  some 
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objects  more.  Furthermore*  not  all  die  spatial  frequency  /orientation  pairs  of  spectral  information  in  a  7  by  7  matrix  will 
be  needed  for  all  objects.  This  fact  should  be  evident  from  looking  at  the  various  magnitude  spectra.  The  7  by  7  matrix 
should  be  viewed  as  a  general  box  size  into  whose  slots  the  relevant  spectral  components  from  objects  are  put. 

Rate-distortion  theory  (Shannon,  1960)  is  a  field  that  attempts  to  deal  with  information  that  has  to  be  transmitted  over 
a  noisy  channel  with  finite  error.  It  is  suggested  that  the  spatial  filtering  typified  in  this  dissertation  may  lead  to  the 
value  of  D,  the  measure  needed  to  specify  the  average  distortion  of  an  object.  Such  a  measure  is  required  to  specify  the 
cost  of  errors  in  a  communication  system. 

19.4.12  Physiological  Evidence 

The  physiological  and  psychophysical  evidence  which  was  used  to  support  and  to  specify  the  spatial  filters  of  this 
dissertation  has  been  discussed  in  Chapters  5-7. 

19.4.13  Summary 

The  previous  discussions  have  shown  briefly  how  the  filter  concept  can  begin  to  offer  certain  explanations  to  the  major 
problems  of  pattern  recognition.  The  filter  concept  has  attempted  to  go  beyond  explanations  of  certain  individual 
problems  to  provide  a  quantitative  structure  and  concept  that  appears  to  unify  the  different  classes  of  certain  perceptual 
problems— Gestalt  laws,  texture  analysis,  visual  illusions,  and  multistability.  Furthermore,  monocular  depth 
perception  and  memory  have  been  discussed,  as  well  as  memory  within  the  concept  of  filtering.  Although  machine 
pattern  recognition  of  three-dimensional  solid  objects  and  overlapping  objects  has  not  been  attempted  so  far  using 
these  kinds  of  techniques,  it  does  appear  that  they  may  be  equally  solvable  within  the  context  of  the  filtering  presented 
in  this  dissertation. 

19.5  Why  Are  the  Low  Spatial  Frequencies  More  Important  than  the  High  Spatial  Frequencies? 

Objects  can  still  be  correctly  identified  from  just  the  high  spatial  frequencies.  If  so,  then  why  emphasize  the  importance 
of  the  low  spatial  frequencies?  Four  reasons  for  the  importance  of  low  spatial  frequencies  over  the  high  spatial 
frequencies  in  the  processing  of  spatial  information  are  given  in  the  following  paragraphs. 

First,  it  has  been  shown  that  only  a  small  number  of  spatial  frequencies  are  needed  to  provide  a  quantitative  description 
of  the  basic  form  of  an  object.  The  minimum  number  of  spatial  frequencies  required  to  correctly  classify  an  object 
provides  a  firm  criterion  for  their  selection,  whereas  higher  spatial  frequencies  do  not  (e.g..  see  Section  9.4).  There  is  a 
large  set  of  possible  high  spatial  frequencies  that  could  be  selected  that  car>  satisfy  the  criteria  of  representing  the  object. 
Also,  any  set  of  high  spatial  frequencies  would  have  Co  contain  a  larger  number  of  spatial  frequencies  than  the  minimum 
set  of  low  spatial  frequencies.  For  example,  a  bandwidth  of  two  octaves  with  a  centre  frequency  of  2  would  have  spatial 
frequencies  for  2  to  8.  or  7  integer  spatial  frequencies.  This  would  increase  the  memory  store  in  either  a  machine  or  a 
biological  system. 

Second,  most  of  the  spectral  energy  is  found  in  the  low  spatial  frequencies  of  objects.  This  was  shown  even  for  outline 
.sfcjerls.  It  is  not  too  surprising  that  the  spectral  region  containing  most  of  the  energy  of  an  object  also  contains  most  of 
the  important  form  information  about  the  object. 

Third,  it  has  been  shown  repeatedly  that  the  low  spatial  frequencies  provide  the  basic  form  of  an  object  independent  of 
the  particular  features  that  make  up  the  form.  The  high  spatial  frequencies,  although  able  to  convey  the  basic  form 
implicitly,  do  not  solve  the  problem  of  generalization;  another  stage  of  processing  would  have  to  be  used  to  extract  the 
basic  form  from  such  high  spatial  frequencies. 

Fourth,  any  serious  attempt  to  understand  how  the  human  visual  system  processes  the  objects  it  sees  must  contend  with 
the  spatial  information  that  is  lost  under  everday  viewing  conditions.  The  low  spatial  frequencies  contain  the  only 
spatial  information  that  can  survive  reduced  illumination,  low  contrast,  minimum  retinal  sampling  from  distance 
viewing,  inhomogeneous  retinal  sampling  from  close  viewing,  movement,  etc. 
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Of  course,  the  high  spatial  frequencies  play  an  important  role  in  providing  information  about  the  details  of  objects 
without  which  some  discriminations  would  be  impossible.  However,  from  these  practical  and  realistic  considerations,  it 
is  the  low  rather  than  the  high  spatial  frequencies  that  appear  to  contain  the  important  general  information  of  spatial 
objects  for  machines  and  biological  visual  systems. 

19.6  What  Is  the  Bandwidth  and  Shape  of  the  Channels? 

The  reader  may  expect  one  specific  bandwidth  and  filter  shape  to  come  out  of  this  study.  But  that  is  not  the  result. 
Several  detailed  examples  of  filtered  images  have  shown  that  similar  forms  can  exist  from  a  general  band  of  several 
harmonics  filtered  with  different  functions.  This  is  not  a  weakness  of  the  filter  concept  in  an  attempt  to  understand 
visual  perception  but,  rather,  a  strength.  What  the  filtering  results  suggest  is  that  a  large  amount  of  spatial  information 
can  be  extracted  from  objects  without  the  need  to  invoke  complex  mechanisms  to  extract  and  manipulate  precise 
features  from  objects. 

The  previous  discussions  (Chapters  6  and  71  of  the  bandwidths  of  channels  from  neurophysiological  and  psychophysical 
data  suggest  a  certain  continuum  of  both  bandwidth  and  shape.  This  kind  of  a  continuum  is  what  one  would  expect 
from  a  biological  system.  It  is  suggested  that  any  serious  theory  about  visual  information  processing  must  contend  with 
spatial  information  in  such  a  continuum  of  bandwidths  and  shapes  of  channels. 

19.7  What  Could  Be  the  Basis  for  the  One-  to  Two-Octave- Wide  Channels? 

There  is  a  common  factor  to  the  biological  data  (see  Chapters  6  and  7),  machine  pattern  recognition  techniques  (see 
Section  19.31.  and  the  filtering  that  has  been  presented  in  this  dissertation.  All  three  systems  extract  spatial  frequency 
(size)  and  orientation  information  in  a  two-dimensional  space.  Furthermore,  using  neurophysiological  and 
psychophysical  techniques,  one-  to  two-octave  bandwidths  have  been  found  repeatedly  for  channels  in  biological 
systems.  Similar  bandwidths  have  been  found  to  provide  successful  machine  pattern  recognition  when  the  algorithms 
were  based  upon  filtered  transforms.  Finally,  it  has  been  shown  often  by  the  author,  using  both  optical  and  digital 
filtering  techniques,  that  a  channel  bandwidth  of  one  to  two  octaves  can  provide  information  about  the  basic  forms  of 
many  objects. 

Is  the  similar  channel  bandwidth  that  has  been  found  to  be  important  for  processing  objects  in  three  different  systems 
a  happy  coincidence,  or  does  it  reflect  some  basic  unit  of  spatial  information?  Further  research  is  needed  to  answer  that 
question  fully.  It  is  suggested,  however,  that  the  one-  to  two-octave  bandwidth  does  represent  a  fundamental  unit  of 
spatial  information  processing  in  the  visual  system. 

It  seems  that  the.  roughly,  two  to  four  harmonics  that  make  up  the  one-  to  two-octave  channel  bandwidth  found  in 
mammalian  visual  systems  reflect  some  basic  physiological  substructure.  It  also  seems  reasonable  that  this  unit  is  found 
at  the  earliest  processing  of  spatial  information  in  the  visual  system.  Once  the  optical  image  is  filtered  by  the  retinal 
ganglia,  no  more  information  about  the  local  (explicit,  see  Section  19.4.21  spatial  information  of  an  object  is  available  to 
higher  visual  processes.  Thus,  implicit  (see  Section  19.4.2)  phase  information  about  an  object  would  remain  and  be 
limited  by  the  extent  of  overlap  of  receptive  fields  in  the  retina.  Hobson  (1975)  determined  that  the  number  of 
overlapping  receptive  fields  in  any  one  area  of  the  retina  of  cat  is  about  three. 

Further  supportive  evidence  for  the  one-  to  two-octave  channel  bandwidth  comes  from  the  data  of  Wilson  and  Bergen 
( 19771.  They  compared  two  sets  of  data  collected  to  determine  the  spatial  inhomogeneity,  local  variation,  and 
probability  summation  over  the  retina.  Analysis  of  the  line-spread  function  and  thresholds  taken  at  different  parts  of 
the  retina  suggested  that  there  might  be  just  four  different  receptive  field  sizes  at  each  retinal  eccentricity. 

It  should  be  stressed  that  these  data  do  not  mean  that  there  are  narrow  one-  to  two-octave-wide  channel  bandwidths  at 
the  retina.  They  mean  that  the  local  spatial  information  is  collected  by  relatively  wideband  channels  in  clumps  of  three 
or  four  receptive  fields.  What  is  suggested,  then,  is  that  as  the  spatial  information  is  filtered  into  narrower  channels  at 
the  higher  stages  of  visual  processing,  the  integrity  of  the  initial  implicit  phase  is  maintained  to  provide  a  fundamental 
unit  of  up  to  about  four  harmonics  of  spatial  information. 

Are  there  any  other  clues,  in  addition  to  the  results  of  spatial  filtering,  that  also  suggest  that  spatial  processes  are  limited 

9H 


to  about  four  harmonics?  Atkinson,  Campbell,  and  Francis  ( 1976)  found  that  the  limit  for  counting  linear  arrays  of 
dots  or  lines  with  only  one  glance  (ISO  msec  presentation  time)  with  total  accuracy  is  from  2  to  4  dots,  depending  upon 
the  dot  spacing.  In  a  further  study,  Atkinson,  Francis  and  Campbell  (1976)  reported  that  if  the  stimuli  have  large 
differences  in  orientation  (90  degrees)  and  a  large  intergroup  space,  three  times  the  interdot  interval,  then  accurate 
counting  within  each  subgroup  can  take  place. 

These  results  suggest  that  spatial  information  used  in  counting  is  limited  to  the  initial  bandwidth  of  mechanisms  in  the 
visual  system.  Large  numbers  of  dots  would  require  a  larger  bandwidth  to  provide  sufficient  sampling  for  a  counting 
mechanism. 

After  learning  of  these  results,  the  author  went  back  to  some  early  filtering  on  4  by  4  dot  arrays  (Figure  95,a).  The 
fundamental  frequency,  fj,  of  the  dots  is  32/14,  or  2.29  cycles  per  array  (c/a);  f2  =  4.57  c/a;  f3  =  6.86  c/a;  and  f]  = 

9. 15  c/a.  Therefore,  there  should  be  discriminably  different  dots  from  a  spatial  filter  whose  bandwidth  is  from  one  to 
two  octaves  —  in  this  case,  from  4.5  c/a  to  9. 15  c/a.  The  results  of  the  spatial  filtering  are  shown  in  Figure  95,b,c,d;  the 
dots  are  clearly  discriminable  when  the  third  harmonic  of  the  array  is  captured  by  a  7  by  7  spatial  filter,  6.86  c/a 
( Figure  95,c ).  The  mathematics  required  to  relate  the  bandwidth  required  to  discriminate  the  number  of  elements  in  an 
array  is  given  by  the  array  theorem,  e.g.,  see  Hecht  and  Zajac  ( 1974).  (Briefly,  the  Fourier  transform  of  an  array  of 
similar  objects  is  given  by  the  product  of  the  Fourier  transform  of  an  array  of  point  sources  by  the  Fourier  transform 
of  the  elements. ) 

19.8  What  Is  the  Minimum  Number  of  Spatial  Frequencies  Necessary  for  the  Identification  of  Objects? 

There  is  no  one  number.  Each  task  and  each  object  requires  slightly  different  numbers  of  spatial  frequencies  or 
harmonics.  For  example,  to  detect  the  presence  of  an  object  requires  only  very  low  spatial  frequency,  less  than  the 
fundamental  spatial  frequency  of  the  object.  The  classification  of  Snellen  letters  requires  from  about  1 .5  to  2.5  cycles 
per  letter.  The  identification  of  a  face  from  a  small  class  of  faces  requires  about  4  cycles  per  face  width.  The  number  of 
spatial  frequencies  (referenced  to  the  overall  size  of  the  object)  which  are  required  will  be  based  on  the  part  of  the  object 
required  to  do  the  particular  task.  For  example,  if  one  needs  to  discriminate  a  square  wave  from  a  sine  wave,  then  the 
detection  of  the  third  harmonic  is  sufficient  (Campbell  and  Robson,  1968). 

In  optics,  the  number  of  spatial  frequencies  required  to  allow  objects  to  be  classified  and  discriminated  was  suggested  to 
be  the  half-power  point  of  the  central  maximum  of  the  particular  feature  relevant  for  the  particular  task  (Ginsburg. 
1971,a.b).  The  half-power  point  in  terms  of  discrete  harmonics  of  an  object  is  about  the  first  harmonic.  That  is,  the 
detection  of  the  fundamental  harmonic  of  the  particular  feature  necessary  to  perform  the  specific  task  appears  to  be 
sufficient.  Additional  spatial  information  will  be  redundant  for  that  task  for  a  certain  percentage  of  the  time.  This 
assertion  is  supported  by  an  experiment  performed  by  Quine  ( 1971 )  showing  that  a  notch  contained  in  a  filtered  image 
of  a  large  square  was  correctly  identified  at  the  75  percent  correct  level  when  the  half-power  point  of  the  notch  was 
passed  by  the  filter. 

19.9  The  Roles  That  Fourier  Analysis  Can  Play  in  Spatial  Vision 

1 9.9. 1  Fourier  Analysis  as  a  Metric 

Both  the  theoretical  and  practical  reasons  for  using  Fourier  analysis  as  a  means  to  describe  spatial  information  in  simple 
and  complex  objects  have  been  discussed. 

Furthermore,  Fourier  transformation  has  provided  the  features  of  objects  that  have  been  used  to  yield  a  measure  of 
similarity  of  objects  for  human  and  machine  experiments  (Section  19.31.  Although  there  are  other  transforms  that  could 
be  used,  the  vast  literature  on  the  Fourier  transform,  the  simplicity  of  obtaining  the  Fourier  transform  by  means  of 
optical  diffraction,  and  the  general  availability  of  digital  computer  programmes  for  the  Fourier  transform  have 
motivated  its  use  in  this  dissertation. 

19.9.2  Fourier  Analysis  as  a  Tool  for  Spatial  Filtering 

The  two-dimensional  Fourier  transform  has  been  used  to  analyze  objects  with  respect  to  spatial  frequency  and 


orientation  to  allow  selective  filtering  along  those  stimulus  dimensions.  Very  simple  spatial  filters  were  used  to  extract 
certain  regions  of  spatial  frequency  and  orientation  information. 

It  is  emphasized  that  all  the  filtering  that  has  been  done  using  band-pass  spatial  filters  in  the  Fourier  domain  could  be 
done  by  convolving  the  object  with  the  Fourier  transform  of  the  band-pass  spatial  filters.  Once  the  problem  is  solved  in 
a  linear  transform  domain,  then  it  is  very  simple  to  apply  that  solution  to  the  space  domain.  However,  besides  being 
more  complicated,  convolution  techniques  do  not  offer  insight  into  thinking  about  the  filtering  process  by  mechanisms 
that  range  in  size  and  orientation  that  a  two-dimensional  transform  would,  i.e.,  the  magnitude  and  phase  spectra  of  the 
Fourier  transform. 

19.9.3  The  Fourier  Transform  as  a  Model  of  Visual  Information  Processing 

The  author  has  gained  much  insight  into  two-dimensional  spatial  filtering  from  the  two-dimensional  Fourier  analysis, 
making  an  analogy  between  Fourier  analysis  and  the  mechanisms  in  the  visual  system.  Thus,  the  author  has  used 
Fourier  analysis  as  a  model  of  encoding  visual  spatial  information  by  mechanisms  tuned  to  different  spatial  frequencies 
(sizes)  and  orientation.  The  model  aspect  of  the  Fourier  analysis  will  now  be  discussed. 

There  appears  to  be  a  good  relationship  between  the  filtering  properties  of  the  sinusoids  of  the  two-dimensional  Fourier 
transform  and  visual  mechanisms.  Note  the  weighting  functions  of  the  sinusoidal  filters  of  the  transform  (Figure  2).  As 
can  be  seen  in  the  upper  right  comer  of  each  sinusoid,  the  schematic  weighting  functions  have  similar  properties  to 
certain  receptive  fields  described  in  terms  of  feature  detectors,  e.g.,  Barlow,  et  al.  (1972).  The  cosine  harmonics  have 
filtering  properties  of  line  detectors.  The  sine  harmonics  have  the  filtering  properties  of  edge  detectors.  Such  sinusoids 
could  even  be  called  fuzzy  bar  or  edge  detectors. 

In  general,  these  finite  sinusoids  have  similar  periodicity  to  the  line  weighting  functions  of  complex  cells  in  the  visual 
cortex  of  cat  found  by  Glezer,  et  al.  ( 1973),  and  Pollen  and  Ronner  (1975).  Although  the  shape  is  similar  la  closer  fit 
would  be  obtained  by  tapering  the  sinusoids  with  a  cosine  function),  there  is  no  evidence  for  such  narrowly  tuned 
neurons.  However,  the  two-dimensional  sinusoids  have  provided  an  intuitive  jump  between  a  Fourier-like  transform 
and  cortical  mechanisms  tuned  to  different  ranges  of  spatial  frequency  and  orientation. 

Thus,  two-dimensional  analysis,  typified  by  the  Fourier  transform,  can  provide  not  only  a  tool  but  also  an  analogue 
from  which  to  view  the  decomposition  of  any  object  into  the  same  stimulus  domain  or  feature  space  that  has  been  found 
relevant  from  biological  data— spatial  frequency  (size)  and  orientation. 

19.9.3. 1  From  a  Local  to  a  Global  Harmonic  Analysis 

There  is  a  more  subtle  aspect  of  the  magnitude  spectrum  that  appears  to  be  particularly  relevant  to  the  analysis  of  such 
objects  as  the  multistable  triangles.  That  is,  the  magnitude  spectrum  represents  the  spectral  decomposition  of  all  the 
patterns  within  the  pattern  space  of  analysis  or  aperture.  In  the  case  of  the  triangles,  all  the  triangles  in  the  picture 
constitute  the  pattern  space.  Therefore,  any  one  spectral  region  that  is  filtered  in  the  magnitude  spectrum  will  affect 
each  pattern  similarly,  as  was  shown  with  the  triangles  (Figures  76  and  77).  Although  the  end  result  of  such  a  process 
appears  to  be  an  analogue  to  our  own  perception,  it  does  represent  additional  mechanisms,  or  at  least  interconnections 
between  known  mechanisms,  that  go  beyond  our  present  knowledge  of  visual  processes. 

Consider,  as  an  example,  the  analysis  of  each  triangle  by  known  visual  mechanisms.  Each  triangle  excites  mechanisms 
of  similar  spatial  frequency  (size)  and  orientation,  in  different  regions  of  visual  space.  Thus,  each  triangle  can  be 
considered  undergoing  a  local  Fourier-like  transformation.  This  represents  the  first  stage  of  an  actual  Fourier 
transform.  However,  in  order  to  obtain  the  pattern  information  analogous  to  the  magnitude  spectrum,  there  have  to  be 
other  channels  that  pool  or  act  upon  each  local  spectrum  over  similar  ranges  of  spatial  frequency  and  orientation  and 
cover  large  regions  of  visual  space.  Without  such  processes,  it  seems  highly  unlikely  that  all  the  mechanisms  could  act 
in  unison  to  produce  the  global  multistable  perceptions  of  the  triangles  that  were  observed.  Thus,  spatial  information 
may  be  pooled  from  a  local  harmonic  analysis  into  a  more  global  harmonic  analysis  along  the  stimulus  dimensions  of 
spatial  frequency  (size)  and  orientation. 
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Although  such  mechanisms  offer  a  reasonable  explanation  of  the  perception  of  a  field  of  similar  but  randomly  . 
positioned  equilateral  triangles,  the  mechanisms  appear  to  be  limited  to  regions  of  visual  space  that  contain  similar 
pattern  orientation  and  patterns  that  are  close  to  one  another.  As  previously  discussed  in  Section  13.6,  triangles  (in 
particular  spatial  configurations)  do  not  generally  appear  as  a  group  that  changes  orientation  in  unison.  However,  all 
the  triangles  within  a  subgroup  do  appear  to  point  in  similar  orientations,  as  shown  in  Figure  77.  The  local  grouping 
seems  to  depend  upon  the  similarity  and  proximity  of  each  pattern  within  a  region  and,  further,  certain  regions  can  be 
analyzed  somewhat  independently  of  other  regions.  Thus,  the  concept  of  a  local  harmonic  analysis  for  each  pattern 
could  lead  to  a  more  global  harmonic  analysis  at  the  next  processing  level,  determined  by  the  spatial  arrangements  and 
the  similarity  of  the  patterns  at  an  earlier  processing  level.  A  completely  random  arrangement  of  similar  patterns  will 
allow  a  global  transform  to  extend  over  all  of  visual  space.  Structured  arrangements  of  similar  patterns  restrict  the 
transform  grouping  over  regions  of  visual  space  within  which  the  substructure  is  uniform,  e.g.,  the  collinearity  and 
proximity  of  the  triangles  of  Figure  77. 

One  of  the  rules  that  determine  the  extent  to  which  similar  spatial  frequency  and  orientation  information  will  pool  to 
form  a  more  global  region  might  be  the  classical  grouping  concepts  such  as  similarity  and  proximity.  These  concepts 
were  shown  to  be  physically  realizable  from  the  low  spatial  frequencies  of  classical  Gestalt  grouping  patterns  (Section 
9.5).  Perhaps  it  is  the  low  spatial  frequency  images  of  visual  space  that  aid  in  the  formation  of  localized  harmonic 
analyses. 

It  is  from  this  point  of  view  that  the  Fourier  transform  is  an  analytical  tool  as  well  as  a  model  with  which  to  analyze  and 
view  the  way  that  visual  mechanisms  seem  to  be  processing  the  objects  that  are  observed. 

This  analysis,  in  agreement  with  other  studies,  suggests  that  a  two-dimensional  Fourier-like  transform  domain  can 
provide  a  good  first  approximation  to  the  stimulus  domain  in  which  visual  mechanisms  are  processing  the  pattern 
information  (Kabrisky.  1967;  Campbell  and  Robson,  1968;  Ginsburg,  1971;  Pollen,  et  al.,  1971;  Glezer,  et  al.,  1973; 
Maffei  and  Fiorentini,  1973;  DeValois,  1976). 

Of  course,  this  does  not  mean  that  one  should  expect  to  see  a  Fourier  spectrum  at  some  level  of  the  visual  system. 
Rather,  a  sufficient  condition  for  such  a  view  is  that  mechanisms  tuned  to  different  ranges  of  spatial  frequency  (size) 
and  orientation  within  local  regions  of  visual  space  are  used  as  units  of  information  somewhat  independently  of 
mechanisms  tuned  to  different  spatial  frequencies  (sizes)  and  orientations. 

The  term  Fourier-like  is  used  because  there  are  many  other  two-dimensional  transforms  that  would  satisfy  the 
requirement  of  decomposing  patterns  into  different  sizes  and  orientations.  The  decomposition  of  patterns  with  perfect, 
infinitely  long  sinusoids  is  not  a  prerequisite  for  the  preceding  analysis.  Although  a  linear  mathematical  process  has 
been  used  to  create  the  pictures  in  this  dissertation,  the  visual  system  does  not  seem  to  be  strictly  linear.  Indeed, 
previous  results  with  machine  pattern  recognition  and  filtering  have  been  quite  successful  using  other  linear  as  well  as 
nonlinear  two-dimensional  functions  to  transform  and  classify  patterns  ( Kabrisky  and  Carl,  1971 ).  It  appears  that 
almost  any  quasi-periodic  set  of  size  and  orientation  functions  would  suffice  at  this  level  of  analysis. 

The  important  concept  of  this  dissertation  is  filtering  —  not  what  the  particular  filtering  elements  are.  However,  since  a 
particular  transform  must  be  chosen  in  order  for  any  analysis  to  be  done,  one  is  chosen  for  the  best  compromise  between 
being  a  good  tool  and  a  good  model.  For  example,  both  the  Fourier  and  Walsh  transforms  have  been  used  for  pattern 
analysis,  e.g.,  see  Ginsburg  et  al.  (19761.  However,  the  Fourier  transform  has  been  shown  to  provide  more  consistent 
correlates  to  everyday  perception,  e.g..  Gestalt  grouping  and  visual  illusions,  than  does  a  Walsh  transform  (Ragsdale. 

19 1 2).  This  is  not  too  surprising  when  one  considers  that  sinusoids  are  more  like  the  spatial  properties  of  receptive  fields 
than  the  rectangular-shaped  Walsh  functions  (which  are.  incidentally,  literal  bar  and  edge  detectors). 

Again,  the  important  concept  is  not  so  much  the  particular  two-dimensional  transform  components  in  a 
two-dimensional  space.  It  is  unlikely  that  the  use  of  an  analytical  set  of  functions  that  approximate  the  spatial 
properties  of  the  receptive  fields  will,  at  this  level  of  analysis,  substantially  change  the  filtering  results  to  date.  However, 
there  are  sure  to  be  some  phenomena  that  will  be  better  observed  with  a  closer  approximation  to  the  properties  of  the 
receptive  fields. 
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19.10  Future  Investigations 


The  next  stage  to  this  research  is  to  try  to  determine  the  rules  by  which  the  channels  combine.  A  hierarchy  of  filtered 
images  appears  needed  to  provide  both  the  general  and  detailed  spatial  information  about  objects.  The  results  of  some 
of  the  contrast  illusions,  e.g.,  the  subjective  contours,  suggest  that  certain  parts  of  low  band-pass  filtered  images 
"peek”  through  in  places  where  there  is  no  pattern.  Further,  it  is  possible  that  the  classic  problem  of  Mach  bands 
results  from  the  overall  filtered  image  when  the  channels  combine.  This  research  will  continue  based  on  the 
combinational  rules  outlined  in  Section  12.8.3. 

The  author  hopes  to  learn  some  of  the  combinational  rules  from  continued  studies  of  abnormal  vision.  Clearly,  the 
contrast  matching  studies  of  the  amblyope,  discussed  in  Chapter  A-4,  present  a  more  complex  interrelationship 
between  spatial  frequency  and  contrast  than  had  been  previously  considered. 

Another  major  area  that  will  be  investigated  further  is  the  relationship  between  single  and/or  multiple  spatial 
frequencies  of  complex  patterns  and  the  contrast  sensitivity  function  to  predict  the  identification  of  any  arbitrary 
pattern,  especially  for  those  objects  having  a  fundamental  frequency  greater  than  1  c/d.  This  will  require  more  detailed 
examination  of  the  relevant  spatial  frequencies  of  patterns  in  terms  of  their  two-dimensional  spectra  and  the 
mechanisms  in  the  visual  system  that  detect  spatial  stimuli. 

The  hierarchy  of  filtered  images  provides  a  reasonable  construct  that  allows  different  spatial  information  about  an 
object  to  be  attended  to  at  will.  Just  as  either  the  entire  orchestra  or  only  the  woodwind  section  of  the  orchestra  can  be 
attended  to,  similarly  either  the  entire  portrait  or  only  the  form  of  the  portrait  can  be  attended  to.  The  certain  pai  allels 
between  the  visual  and  auditory  systems  suggest  that  the  concept  of  a  hierarchy  of  filtered  images  may  be  relevant  to 
the  auditory  and  other  sensory  systems. 
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SUMMARY 


'This  dissertation  investigated  visual  information  processing  using  spatial  filters  that  are  constrained  by  biological  data. 
The  four  parts  of  this  dissertation  were  ( 1 1  a  theoretical  background  to  Fourier  analysis,  12)  a  review  of  the  literature 
relating  to  the  spatial  filtering  characteristics  of  mammalian  visual  systems,  (3)  visual  information  processing  in  terms 
of  spatial  filtering,  and  14)  the  relating  of  contrast  sensitivity  to  the  identification  of  complex  objects.  The  common 
denominator  in  all  the  investigations  that  have  been  reported  is  spatial  filtering.  The  major  goal  of  the  dissertation  was 
to  attempt  to  explain  certain  aspects  of  visual  information  processing  in  the  visual  system  with  a  filter  concept.  The 
results  of  this  attempt  are  summarized  below: 

1 1 1  Biological  data  from  cat,  monkey,  and  man,  gathered  using  neurophysiological  and  psychophysical  methods, 
provide  much  evidence  for  the  existence  of  mechanisms  tuned  to  different  regions  of  spatial  frequency  and  orientation. 
In  general,  the  spatial  frequency  bandwidth  is  one  to  two  octaves  and  the  orientation  bandwidth  is  +15  degrees.  The 
overall  filtering  characteristic  of  the  visual  system  is  the  envelope  of  the  activity  of  those  mechanisms  and  can  be 
determined  by  contrast  sensitivity  and  contrast  matching  functions. 

(21  Two-dimensional  spatial  filters  were  made  based  upon  the  general  and  specific  filtering  properties  of  the  overall 
( MTF I  and  individual  mechanisms  (channels).  These  spatial  filters  were  used  to  filter  a  wide  variety  of  objects. 

(3)  The  perceptual  problems  of  the  classification  of  simple  and  complex  objects  —  Gestalt  principles,  visual  textures, 
visual  illusions,  and  multistability —  with  few  exceptions,  have  yielded  gracefully  to  the  concept  of  spatial  filtering. 
Parsimony  reigned  as  explanations  for  these  seemingly  disparate  visual  phenomena  were  given  in  terms  of  the  known 
two-dimensional  filtering  properties  of  the  visual  system. 

(41  A  detailed  study  using  generalized  and  biologically  derived  band-pass  spatial  filters  further  demonstrated  the  power 
of  a  general  filter  concept  for  visual  information  processing.  The  full  spectrum  of  perceptual  information  about  a 
complex  object  from  basic  form  to  the  finest  detail  was  available  with  only  seven  two-octave-wide  band-pass  filters 
(channels).  These  results  led  to  the  concept  of  a  hierarchy  of  filtered  images. 

(51  In  general,  the  MTF  filter  provided  the  details  of  objects  that  correspond  to  what  is  seen.  One-  to  two-octave 
band-pass  filters  appeared  to  provide  the  generalized  information  that  is  perceived  about  objects.  The  specific 
bandwidth  and  shape  of  the  spatial  filters  were  not  important  for  the  class  of  objects  and  perceptual  problems  discussed 
in  this  dissertation. 

(<>l  Combining  the  filtering  concepts  of  the  minimum  information  required  for  the  identification  of  objects  with  data 
from  an  experiment  that  determined  the  minimum  amount  of  contrast  required  for  the  identification  of  Snellen  letters 
demonstrated  the  predictive  power  of  the  filter  concept.  The  Snellen  acuity  of  1 7  of  22  eyes  of  amblyopes  and  patients 
suffering  from  multiple  sclerosis  could  be  predicted  within  one  Snellen  line.  The  increase  of  spatial  sensitivity  over  a 
narrow  range  of  spatial  frequency  of  one  amblyope  resulted  in  a  concomitant  increase  in  Snellen  letter  acuity. 

Kspecially  interesting  was  that  the  increased  Snellen  acuity  could  be  explained  from  the  minimum  number  of  spatial 
frequencies  required  for  letter  identification.  These  results  appear  to  provide  the  first  direct  evidence  for  the  existence  of 
narrow-band  mechanisms  in  the  visual  system  which  are  used  in  the  identification  of  complex  objects.  Thus,  the  ability 
to  establish  a  relationship  between  Snellen  letter  acuity  and  contrast  sensitivity  suggests  that  the  filter  concept  is  neither 
blind  nor  empty. 

In  general,  these  results  support  the  view  that  the  MTF  is  the  window  which  reduces  the  range  of  size  of  objects  that 
can  be  seen.  The  much  narrower  and  orientation-selective  filters  that  make  up  the  MTF  appear  to  filter  relevant  form 
information.  The  outputs  of  all  the  individual  filters  appear  to  be  presented  simultaneously  to  create  the  global  picture 
that  is  seen. 

The  picture  that  emerges  from  these  studies  is  that  visual  perception  may  be  structured  from  a  hierarchy  of  filtered 
images,  each  image  representing  one  level  of  visual  information.  This  hierarchy  of  filtered  images  may  be  the  keyboard 
from  w  hich  the  harmony  of  visual  perception  is  obtained. 
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"If  sight  were  to  deceive  us  as  to  the  position  and  distance  of  external  objects,  we 
should  at  once  become  aware  of  the  delusion  on  attempting  to  grasp  or  to  approach 
them.  This  daily  verification  by  our  other  senses  of  the  impressions  we  receive  by 
sight  produces  so  firm  a  conviction  of  its  absolute  and  complete  truth  that  the 
exceptions  taken  by  philosophy  or  physiology,  however  well  grounded  they  may 
seem,  have  no  power  to  shake  it.  ” 

Hermann  von  Helmholtz,  1869 
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62  The  Benussi  Ring  Illusion 

63  Variations  of  the  Kanizsa  Triangle 

64  Contour  Plots  and  Numerical  Printouts  of  Low-Pass  Filtered  Solid  and  Outline  Versions  of  the  Kanizsa 
Triangle 

65  The  Ponzo  Illusion  (Large  Sized  I 

66  Theoretical  and  Experimental  Data  for  the  Magnitude  of  the  Miiller-Lyer  Illusion 

67  Eye  Scan  Paths  from  the  Miiller-Lyer  Illusion  (from  Yarbus.  1%7I 

68  Eye  Scan  Paths  from  a  Portrait  of  a  Young  Girl  (from  Yarbus.  10671 

60  Eye  Scan  Paths  from  a  Bust  of  Nefertite  (from  Yarbus,  1067) 

70  Portion  of  “Slave  Market  with  Apparition  of  the  Invisible  Bust  of  Voltaire"  by  Salvador  Dali 

71  Variations  of  the  Bisected  Circle 

72  Numerical  Printouts  of  Variations  of  the  Bisected  Circle 

73  The  Rubin  Face-Vase  Object 

74  Multistable  Dot  Pattern 

75  Square- W ave  Gratings  at  Three  Orientations 

76  Multistable  Triangles  —  Equilateral  and  Random 

77  Numerical  Printouts  of  Equilateral  Multistable  Triangles  and  Collinear  Multistable  Diamonds 
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Multistable  Triangles  —  Scalene  and  Random 
One-Octave  Band-Pass  Filtered  Portrait 


Two-Octave  Band-Pass  Filtered  Portrait 

One-Octave  Band-Pass  Filtered  Portrait  Having  a  Higher  Centre  Frequency 
One-Octave  Channel  Filtered  Portrait 
Two-Octave  Channel  Filtered  Portrait 

Table  of  First  2.5  Cycles  of  Letters  Associated  with  Each  Snellen  Line  Number 

Synthesis  of  Snellen  Letters 

Snellen  Letter  Acuity  Chart  Used  in  Experiments 

Table  of  Contrast  and  Contrast  Sensitivity  Required  for  Detection  and  Identification  of  Snellen  Letters 

The  Percent  Contrast  Required  for  the  Detection  and  Identification  of  Snellen  Letters  for  Each  Snellen 
Line 

Contrast  Sensitivity  Required  for  the  Deteaction  and  Identification  of  Snellen  Letters  for  Each  Snellen 
Line 

Ratio  of  Contrast  Required  for  Identification  Versus  Detection  of  Snellen  Letters  as  a  Function  of  Snellen 
Line  Number 

Contrast  Sensitivity  Function  of  Subject  A.G. 

Table  of  Cycles  and  Associated  Contrast  Sensitivity  Required  for  the  Identification  of  Snellen  Letters 
Table  of  Predicted  Snellen  Letter  Acuity  from  the  Contrast  Sensitivity  of  10  Subjects 
Measured  Versus  Predicted  Snellen  Letter  Acuity 
Spatial  Filtering  a  4  by  4  Dot  Array- 

Curve  for  Determining  the  Percent  Contrast  from  the  Polarizer  Angle 

Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  J.P. 

Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  M.B. 

Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

Average  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve  of  Figures  W  and  100 

Decrease  in  Difference  Between  Eyes  Curve  for  the  First  Two  Test  Sessions:  Amblyope  A.H. 

Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Effect  of  Abnormal  Retinal 
Correspondence  I  ARC  I 
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Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

105  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

106  Average  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve  of  Figures  104,  105 

107  Decrease  in  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

108  Average  from  Three  Sets  of  Contrast  Sensitivity  F unctions  and  Difference  Between  Eyes  Curves: 
Amblyope  A.H. 

10*1  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

1 10  Decrease  in  Difference  Between  Eyes  Curve  from  the  Initial  to  the  Final  Test  Session:  Amblyope  A.H. 

1 1 1  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

1 12  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

1 13  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Amblyope  A.H. 

114  Contrast  Sensitivity  Versus  Time:  Amblyope  A.H. 

1 15  Average  Contrast  Matching  Functions  for  Both  Eyes  of  Amblyope  A.H. 

1 16  Contrast  Matching  Functions  from  Three  Separate  Test  Sessions:  Amblyope  A.H. 

1 1 7  Difference  Between  Eyes  Curves  for  Each  Contrast  Match  for  Three  Separate  Test  Sessions: 

Amblyope  A.H. 

1 18  Average  Difference  between  Eyes  Curves  for  Three  Contrast  Matches  of  Figure  1 15 

1 1‘>  Contrast  Matching  Functions  over  a  Finer  Spatial  Frequency  Range 

126  Difference  Between  Eyes  Curves  for  Each  Contrast  Match  for  Two  Separate  Test  Sessions,  I  Figures 

1 16a,  bl  and  Their  Average  (Figure  1 19cl 

121  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Multiple  Sclerosis  C.H. 

122  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Multiple  Sclerosis  S.S. 

123  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Multiple  Sclerosis  H.R. 

124  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Multiple  Sclerosis  A.J. 

125  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Multiple  Sclerosis  M.D. 

126  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve:  Multiple  Sclerosis  B.M. 
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FIGURES 


A  brief  explanation  of  symbols  used  in  the  figure  captions  follows. 

MTF  Filters 

MTF  indicates  that  the  spectrum  or  image  has  been  spatially  filtered  using  a  filter  based  on  the  two-dimensional 
contrast  sensitivity  function  shown  in  Figure  16.  MTF(L)  and  MTF(HI  indicate  that  the  spatial  filter  has  different 
amounts  of  low  spatial  frequency  attenuation:  threshold  and  suprathreshold.  The  MTF  filters  are  discussed  in 
Section  8. 1 . 

Band-Pass  Filters 

(U  by  VI  indicates  the  spatial  frequency  bandwidth  of  a  two-dimensional  spatial  filter  in  the  U.  V  transform  domain.  U 
and  V  are  the  number  of  spatial  frequencies  passed  by  the  filter  along  the  U  and  V  axis.  ( U  by  V  -  1 1  is  a  band-pass 
filter  which  passes  U  and  V  spatial  frequencies  minus  the  fundamental  frequency,  f  | .  Thus,  (5  by  SI  means  that  S  times 
f  |  spatial  frequencies  of  the  object  in  the  U,  V  dimensions  have  been  passed  by  the  filter.  Band-pass  filters  are  discussed 
in  Sections  4.2  and  l).  1 . 

Log  Transformation 

An  L  after  the  MTF  or  (U  by  VI  means  that  the  intensity  values  of  the  filtered  images  were  log  transformed  to  increase 
the  visibility  of  the  low  contrast  features  in  the  pictures.  The  technique  of  log  transformation  is  discussed  in  Section  (>.2. 

References  to  Chapters  and  Sections 

The  numbers  in  parentheses  after  the  figure  number  refer  to  the  chapter  and  section  in  the  text  of  Volumes  1  and  11  in 
which  the  figure  is  discussed  in  detail.  Thus  (2.5. 1 1  means  that  the  figure  is  discussed  in  Chapter  2,  Section  5. 1 . 
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(b) 


Figure  <>.  Three-Dimensional  View  of  Two-Dimensional  Ideal  Low-Pass  Kilters 
with  Their  Point-Spread  Functions:  la  I  Circle  Functions;  (bl  Square 
Functions:  1 1 1  Low-Pass  Filters;  121  F'ilter  Point-Spread  F'unctions. 
(Sections 4.2.2,  8.2.  D  front.) 
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Kijriire  H.  A  Sine-Wave  Orating  W  hose  Spatial  Frequency  Is  Changing  logarithmically.  (Section  5. 1 1 

132 
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Figure  10.  A  Sine-Wave  Grating  Whose  Spatial  Frequency  and  Contrast  Are  Changing 
logarithmically.  (Section  .">.1 1 
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Figure  14.  Normalized  Contrast  Sensitivity  Functions  for  0.  •4.">.  'XI  Degrees  from  the 
Averaged  Data  of  Campbell,  kulikowski.  and  (.evinson  (I'Hgd. 

(Section  8. 1 1 
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Figure  I  ”).  Table  of  Digital  Data  for  One  Quadrant  of  a  Two-Dimensional  Contrast  Sensitivity  Function: 
lal  MTFtLI  Filter;  (bl  MTFtHl  Kilter.  iSection  8.1 1 


t 


30 

20  J 


♦ 


-30-20-10  0  10  20  30 


-30-20-10  0  10  20  30  -  30-20-10  0  10  20  30 

O) 

,  •  **  ;!■-  -io 

!  1  !:-il-oo 

■■■■■■■J0--3O 

-30  -20-10  0  10  20  30 
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NORMALIZED  AMPLITUDE 
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DISTANCE 


7.  <  iompariton  <if  the  Killer  ( iharacteristic*  of  the  Sine  Funetion  (Solid  l.inel.  a  Simple  dell  in 
the  Venial  (lorlex  of  ('.at  (Fillet!  C,irele*l.  and  the  Shape  of  a  INyehophysiral  Channel 
Determined  l»\  Adaptation  (Dashed  l.inel.  (Section  H.2. 1 1 


f  inure  IK.  A  Diagram  of  an  Ideal  Ism-Pass  Digital  Transform  Filler  (hat  Passes  the 
First  Four  Harmonies.  I  Section  8.2. 1  I 


1 48 


Figure  I').  Spatial  Filtering  a  Letter  T:  lul  Original  Illusion;  Ibl  MTFlLl 
Filtered  Images:  lei  MTF'lHl  Filtered  Images;  Idl  MTF'lHlI, 
Filtered  Images;  tel  MTF'lH  I  It)  by  (>ll.  Filtered  Images;  Ifl  l/m- 
I’ass  Iti  by  til  Filtered  images;  Igl  Lmv-I’ass  It)  by  (*  —  I  I  F  iltered 
Images;  II  I  Magnitude  Speetra;  (21  F  iltered  letters.  (Sections*).  1 
d.2l  leont.l 


(1)  (2) 


(g) 


Figure  I'*.  Spatial  Filtering  a  LetterT:  (alOriginal  Illusion;  Ibl  MTFlLI 
Filtered  Images;  (el  MTFlHI  Filtered  Images;  Idl  MTFlHlL 
Filtered  Images;  (el  MTFlH  1 1()  by  6IL  Filtered  Images;  If  I  Low- 
Pass  ((>  by  61  Filtered  Images;  Igl  Low-Pass  16  by  6  -  1 1  F  iltered 
Images:  ( 1 1  Magnitude  Spectra;  (21  Filtered  Letters.  (Sections 
6.1.6.21  Icon!.  I 
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Figure  20.  Two-  and  Throe- Dimensional  Contour  IMots  of  a  Filtered  l/Otter  ’I 
lal  Two-Dimensional  Plot  after  Filtering  with  MTFlI.I; 

(hi  Two-Dimensional  Plot  after  Filtering  with  MTFtH  I; 
lelThree-Dimensional  Plot  after  Filtering  with  MTFlLl; 

Idt  Three-Dimensional  Plot  after  Filtering  with  MTF(I,|2; 

(el  Three-Dimensional  Plot  after  Filtering  with  MTFtlD-. 
iSeetions  0.2,  12.3.31  fount.  I 


Figure  21 .  Sequential  Low-Pass  Filtering  of  a  Letter  T  after  MTFlLl  Filtering:  lal  Low- 

Pass  1 1  by  1 1  Filtered  Letter  T;  fbl  I<ow-Pass  12  by  21  Filtered  Letter  T;  Irl  Ixm- 

Pass  (3  by  .31  Filtered  letter  T;  tdl  Low-Pass  14  by  41  Filtered  Letter  T;  (el  Low- 

Pass  13  by  31  Filtered  letter  T:  if  I  Ix>w-Pass  It)  by  f>>  Filtered  letter  T. 

(Section  ‘L2I 
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Figure  21.  Sequential  Low-Pass  Filtering  of  a  Letter  T  after  MTF1LI  Filtering:  (a  I  Ix>w-Pass 
1 1  by  1 )  Filtered  Letter  T;  Ibl  Low-Pass  <2  by  21  F  iltered  letter  T;  (el  I/ow-Pass 
(3  by  31  Filtered  letter  T;  (dl  I/iw-l’ass  (4  by  41  Filtered  letter  T;  tel  l/ow-Pass 
(3 by  31  Filtered  letter  T;  (fl  Low-Pass  (<>  by  (»>  Filtered  letter'!.  (Section  '».2( 
Icont.  I 
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Figure  22.  Sequential  Low- Pass  Filtering  of  a  letter  T  after  MTFIH I  Filtering:  la)  Ism- Pass 
1 1  by  1 1  Filtered  letter  T:  (bl  Low- Pass  12  by  21  Filtered  letter  T;  tel  Ism-Pass 
til  by  ill  Filtered  letter  T;  tdl  Ism-Pass  (4  by  41  Filtered  Letter  T;  (el  Ism-Pass 
til  by  .”>1  Filtered  letter  T;  If  I  Ism-Pass  ((>  by  ()l  Filtered  Letter  T.  (Section  '1.21 
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Figure  22.  S«*<|n«‘ntial  l^m-l’ass  Filtering  of  a  letter  T  after  MTK  III  I  Filtering:  la  I  lxtw-1'aw* 
1 1  by  I  I  Filtered  letter  T;  (bl  lam-Paw  12  by  21  Filtered  letter  T;  let  l/ow-l'awt 
Id  by  .it  Filtered  letter  I’;  Idl  I/OW-IVh*  I  t  by  4t  Filtered  letter  T;  let  l,ow  -I'asa 
Id  by  .”>1  Filtered  letter  I’;  (ft  I/Ow-Pas*  lit  by  <>l  Filtered  letter T.  iSeetion  *1.21 
leont.l 
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Figure  24.  Contour  Plots  of  a  Synthesized  Dot  letter  (J:  tal  lyow-l’ass  1 1  by  1 1  F'iltered 
letter;  tbt  Diw-l’ass  12  by  21  Filtered  letter;  lei  Divv-Pass  13  by  3)  F  iltered 
l-etter;  idl  Divv-I’ass  14 by  41  Filtered  letter;  lei  Diw-I’ass  (3  by  31  Filtered 
letter:  Ifl  l.ow-1'ass  Hi  by  (il  F  iltered  letter;  Igl  Diw-I’ass  IT  by  Tl  F  iltered 
letter.  (Seetion  H.4I 
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Figure 24.  Contour  Plots  of  a  Synthesized  Dot  letter <J :  (al  lxm-l’ass  (1  by  II  Filtered 
letter;  tbl  l>owr-Pas«  12  by  21  Filtered  letter;  lei  Low- Pass  (3  by  111  Filtered 
letter;  Idl  Diw-Pass  14  by  41  Filtered  letter;  lei  Low-Pass  (!>  by  ol  Filtered 
letter;  If  I  Iziw-Pass  lb  by  hi  Filtered  letter;  Ijtl  D>w-Pass  IT  by  Tl  Filtered 
letter.  I.Seetion  *1.41  leont.l 
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( 'notour  I’lols  of  a  llich-l’aw  Filtered  Dot  I, otter  (j.  There  Are  l(>  Spatial 
Harmonies  anil  the  Ton  Spatial  Frequeneies  Are  Sequential!)  Removed  ( file  at 
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Figure  23.  Contour  Plots  of  a  High-Pass  Filtered  Dot  Letter  G.  There  Are  If)  Spatial 

Harmonics  and  the  Low  Spatial  Frequencies  Are  Sequentially  Removed  One  at 
a  Time:  (al  High-Pass  (-1  by  1 )  Filtered  Letter;  Ibl  High-Pass  1-2  by  21  Filtered 
L/etler;  Icl  High-Pass  1-3  by  31  Filtered  Letter:  (dl  High-Pass  1-4  by  41  Filtered 
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Figure  2<>.  Similar  Forms  from  Ism  Spatial  Frequencies of  Four  Discriminahly  Different  I-etterfi  s: 

(a  I  Full  (i:  (1)1  Dot  (i;  Id  Serif  (r:  Id  I  Slash  <»;  1 1  M  >ri(f inal  Fetters;  1 2 1  Ism -Pass  (  I  by  41 
Filtered  Fetters.  (Section  ‘>.4. 1 1  Icont.l 
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Figure  27.  An  Example  of  Closure  from  the  Low  Spatial  Frequencies  of  a  Discrete  Line  Letter  H: 

(al  Original  Object:  Ibl  Low-Pass  (4  by  41  Filtered  Magnitude  Spectrum;  Icl  Low-Pass  14  by 
41  Filtered  Image.  (Section ‘>.61 
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(bl  MTF  Filtered  Array.  (Section  1 1.3.21  leont.l 
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Figure  40.  Examples  of  Letter  Arrays  Used  in  Beck's  I  I'Wibbl 
Similarity  Grouping  Experiment.  (Section  1 1 . .4.21 
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Figure  41.  Numerical  Printouts  of  MTFlH)  Filtered  Pairs  of  letters:  lalT-X; 
Ib)  T-Tilted  T.  ISeetion  1 1.3.31  leont.l 
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Figure  46.  The  Poggendorff  Illusion:  la)  Standard  Illusion;  Ibl  Obtuse  Angles  Only  Illusion;  let  Acute  Angles  Only  Illusion; 

1 1 1  Original  Illusions;  (21  MTF  Filtered  Magnitude  Spectra;  (3)  MTF  f  iltered  Illusions;  141  MTFI5  by  3IL  Filtered 
Illusions;  (5)  Low-Pass  (5  by  51  Filtered  Illusions.  (Section  12.21  (corn.) 
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Numerical  Printouts  of  the  Filtered  PotfKendorff  Illusion:  (a  I  MTF  Filtered 
Illusion;  Ibl  MTF  (.">  by  51  Filtered  Illusion;  tel  Inverse  MTF  I  MTF*  1 1  Filtered 
Illusion.  (Section  1 2.2. 1 1 
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Figure  4‘).  Numerical  Printouts  of  an  Illusion  Due  to  Error  in  Aligning  Edges:  (a  I  Original 
Illusion;  lb)  MTF  Filtered  Illusion.  (Section  12.2.21 
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Figure  .‘>.'5.  The  W  Degree  Configuration  of  the  Horizontal- Vertical  Line  Illusion: 

la  I  Original  Illusion:  (bl  MTFlLl  Filtered  Images;  (cl  MTFlH  I  Filtered 
Images:  Idl  MTFlH  IL  Filtered  Images;  lei  MTFlH  I  It)  by  bl  Filtered 
Images:  1 1 1  Filtered  Magnitude  Spectra:  121  Filtered  Illusions. 

I  Section  1 2.,' \ 25 1 
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I1  igure  .>3.  The  (K)  Decree  Configuration  of  the  Horizontal-Vertical  Line 
Illusion:  la) Original  Illusion:  IblMTFlLI  Filtered  Images: 
Id  MTFlHl  Filtered  Images;  Idl  MTFlHIL  Filtered  Images; 
lei  MTFlH  I  !(>  by  ()l  Filtered  Images;  1 1 1  Filtered  Magnitude 
Spectra:  121  Filtered  Illusions.  iSection  12.3.31  Icont.l 
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Figure  .V>.  Simple  Line  Illusions:  la  I  Illusion  One:  Ibl  Illusion  Two: 
(cl  Numerical  Printout  of  MTF  Filtered  Illusion  One: 

ld)  Numerical  Printout  of  MTF  Filtered  Illusion  Two; 

le)  Iso-Contour  Plots  of  MTF  Filtered  Illusion  One; 

(fl  Iso-Contour  Plots  of  MTF  Filtered  Illusion  Two. 
(Section  12.3.41 
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Figure  .V».  Simple  Line  Illusions:  la)  Illusion  One;  Ibl  Illusion  Two; 
Id  Numerical  Printout  of  MTF  Filtered  Illusion  One; 

Id  I  Numerical  Printout  of  MTF  Filtered  Illusion  Two; 
le)  Iso-Contour  Plots  of  MTF1  Filteared  Illusion  One; 

If  I  Iso-Contour  Plots  of  MTF  Filtered  Illusion  Two. 
(Section  12.3.4)  Icont.  I 
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Figure  .V).  Simple  Line  Illusions:  (a)  Illusion  One:  lb)  Illusion  Two; 

lc)  Numerical  Printout  of  MTF  Filtered  Illusion  One: 

ld)  Numerical  Printout  of  MTF  Filtered  Illusion  Two; 

le)  Iso-Contour  Plots  of  MTF  Filtered  Illusion  One; 

(fl  Iso-Contour  Plots  of  MTF  Filtered  Illusion  Two. 
(Section  12.3.41  (cont. I 
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Figured).  Simple  Line  Illusions:  lal  Illusion  One;  (bl  Illusion  Two; 
(cl  Numerical  Printout  of  MTF  Filtered  Illusion  One: 

(dl  Numerical  Printout  of  MTF  Filtered  Illusion  Two; 

(el  Iso-Contour  Plots  of  MTF  Filtered  Illusion  One; 

(f  I  Iso-Contour  Plots  of  MTF  Filtered  Illusion  Two. 
(Section  12.3.41  (cont.  I 
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Figure  38.  Numerical  Printouts  of  an  Illusion  of  Misjudged  Distance:  la)  Original 
Illusion;  lb)  MTF  Filtered  Illusion.  (Section  12.4.2) 
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Figure  .VI.  Numerical  Printouts  of  an  Illusion  of  Misjudged  Distance:  (a)  Original  Illusion; 
Ibl  MTF  Filtered  Illusion.  (Section  12.4.21  Icont.) 
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Figure  5*).  The  Ziillner  Illusion:  (a)  Original  Illusion;  (bl  MTF  Filtered  Illusion;  Ic)  Low-Pass 
(4  by  41  Filtered  Illusion.  (Section  12.5) 
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Ki^urt' Ml.  The  Hermann  (irid  Illusion:  la  I  Original  Illusion:  Ibl  Digitized  Version  of 
Illusion;  lei  MTF  Filtered  Illusion.  (Section  12.1).  1 1  (eont.l 
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Figure  W).  The  Hermann  Grid  Illusion:  (a)  Original  Illusion:  lb)  Digitized  Version  of 
Illusion;  I  cl  MTF  Filtered  Illusion.  (Section  12.6.1 1  (cont.l 
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Figure  (>2.  The  Benussi  Ring  Illusion:  ial  Original  Illusion:  Ibl  Illusion  with 

Different  Contrast;  If  I  b  with  a  White  Line:  Ibl  b  with  a  Black  Line; 

(el  Numerical  Printout  of  Original  Illusion:  (fl  Numerical  Printout  of 
Low-Pass  lit  by  51  Filtered  Illusion;  Igl  Low-Pass  (5  by  51  Filtered  Illusion 
without  Sine  Function  Smoothing;  (hi  High-Pass  (5  by  51  Filtered 
Illusion  with  Sine  Function  Smoothing:  1 1 1  Lnfiltered  Illusions;  121 
Ixiw-Pass  (5  by  51  Filtered  Illusions;  (ill  MTF  Filtered  Illusion. 

(Section  12.h. ill  (font.  I 
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THEORETICAL  AND  EXPERIMENTAL  DATA  FOR 
THE  MAGNITUDE  OF  THE  MULLER -LYER  ILLUSION 
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Figure  Mi.  Theoretical  and  Experimental  Data  for  the  Magnitude  o{  the  Miiller-Lyer  Illusion.  (Section  12.8.1 1 


Figure  <18.  Fte  Srun  Paths  from  a  Portrait  of  a  Young  (iirl  (front  ^  arfms. 

I'KiTI:  (al  Original  Portrait:  ll)l  Fte  Sean  Paths;  (cl  Digitized 
Portrait;  Idl  I.on-Pass  1 24  fit  241  Filtered  Portrait;  (cl  Din-Pass 
ll(i  fit  1  (i I  Filtered  Portrait.  (Section  12.8.21 


(a)  (h) 


Figure  TO.  Portion  of  "Slav**  Market  with  Apparition  of  the  Invisible  Bust 
of  Voltaire”  In  Salvador  Dali:  lgl( )r  initial  Picture;  iblt  triiri- 
nal  Picture  I  ligiti/.ed  to  128  bv  I  28  Pels:  (cl  Ism -Pass  (21 
bv  24 1  Filtered  Picture:  Id  I  Ism-Pass  l8bv  81  Filtered  Picture; 
lei  Bust  Fnlarued  to  Two  Times  the  Original:  ifl  Ism-I’ass  <24 
bv  241  F  iltered  Picture;  (gl  Ism-Pass  I  i<>  bv  l(>l  Filtered 
Picture:  till  Ism-Pass  18  bv  8|  Filtered  Picture.  (Section  14.21 
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Future  7)1.  Portion  of  "Slave  Market  with  Apparition  of  the  Invisible  Hus! 

of  \  oltaire  bv  Salvador  Dali:  lal  < >r itriiiit)  Picture;  ibl  ( Iritti- 
nal  I’ietnre  I  )iiriti/ed  lo  UK  bv  I  JK  I  Vis;  let  Low  -  Pass  I J  I 
bv  J  I  I  Filtered  I’ielure;  id  I  I  ,ov\  -Pass  IK  bv  Hi  Filtered  Piet  lire; 
lei  Must  Fnlartted  to  Tun  Times  the  ( tricinal:  l f  I  Ixiw-Pass  Ijl 
bv  Jit  Filtered  Pielure;  l^l  low -Pass  I  |l>  |>v  |til  Filtered 
Pielure;  dll  I  ow-Pass  i)i  bv  HI  Filtered  Pirtnre.  I  Seel  ion  l.LJI 
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Figure  73.  The  Rubin  Face-Vase  Object:  la)  Original  Object;  (bl  Low-Pass 
18  by  HI  Filtered  Images;  (c)  Low-Pass  ((>  by  61  Filtered  Images; 
Id)  Low-Pass  14  by  41  Filtered  Images;  (el  and  III  Two  Variations 
of  the  Rubin  Face-Vase  Object;  1 1 1  Magnitude  Spectra; 

(21  Filtered  Images.  (Section  13.4) 


(1) 


(2) 


Killin'  7d.  Thi*  Itubin  Fare- Vaae  Object:  (al  Original  Object:  (bl  bm-l’awi 
IH  by  HI  Filtered  Images;  (cl  laiw-Pasa  Hi  by  (>l  Filtered  Images: 
(dl  biH-l‘as.«  II  by  II  Filtered  Images;  lei  and  If  I  Two  \  ariatinn* 
nf  tfie  Rubin  Face-  Vase  Object;  1 1 1  Magnitude  Spectra: 

121  Filtered  Image*.  I  Sect  inn  Id.  II  front.  I 


_>i:. 


Figure  73.  The  Rubin  Fare-Vase  Object:  (a!  Original  Object:  Ibl  Low-I’ass 
IK  by  81  Filtered  Images:  lei  Ix>w-Pass  ((>  by  (>l  Filtered  Images: 
Idl  Ixiw-Pass  14  by  41  Filtered  Images;  lei  and  If  I  Two  Variations 
of  the  Rubin  Face- Vase  ( tbject;  1 1 1  Magnitude  Spectra; 

121  Filtered  Images.  (Section  13.41  Icont.l 


Kiitur**  74.  Multistable  Dot  I’atti'rn:  ta  I  ( triginal  Pattern;  (hi  MTKtI.I 

Filtereil;  Id  Ixm-I’asslli In  (il  Filtered.  Shifted  (-0.01  Harmonic; 
Idl  hm-l’asslldn  (il  Filtereil.  Shifted  1-7.01  Harmonics;  lei  lam 
Pass  10  b>  111  Filtered.  Shifted  10, -til  Harmonics;  Ifl  I ,o\\ -Pass 
Hi  b\  lil  Filtered.  Shifted  <0.-71  Harmonics:  1 1  I  Filtered 
Magnitude  Siicctrn;  121  Filtered  Dot  Patterns.  iSection  1.4. Til 
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r  igure  78.  Multistable  Triangles  —  Scalene  and  Random:  la  I  Original 

Images;  Ibl  Low-Pass  (24 by  24)  Filtered  Images;  Ic).  Id),  and 
lei  Low-Pass  124  by  24)  Filter  Shifted  over  the  Magnitude 
Spectrum  to  Filter  Three  Different  Combinations  of  Two 
Channels;  <1 1  Magnitude  Spectra:  (21  Triangles.  (Section  13.61 
Icont.  I 


Fillin'  7‘).  One-Octave  Band-Pass  Filtered  Portrait.  f(.  =  ( 
Frequency  of  Filter:  lal  Original  Portrait;  Iblf, 
id  I  f(.  =  4;  lei  f(.  =  8:  Ifl  f,.  =  Ui;  Igl  f(.  =  .42;  (h 
(Section  14.41 


Figure  HO.  Two-Oetave  Kami-Pass  Filtered  Portrait,  f,  =  Centre  F  requeue): 

taM  >ri(iinal  Portrait:  tl)l  f(-  — -  I :  tel  f,.  —  2:  id  I  f(-  =  V;  tel  f,.  =  H: 
Ifl  f(.  —  |();  tf|  f(.  =  :I2;  till  f(.  =  f ft .  tSeetioii  I1.  il 


1 


Figure  HI .  <  >ne-<  tetave  Itaml-I’ass  Filtered  1’orlrnit  Having  a  Higher  Centre 
F  requeue).  f(.  =  Centre  Frequeney  of  Filter:  (a I  Original  Portrait; 
lid  f(.  =  2:  Id  f(.  =  4.8;  Id  I  f,.  =  <);  let  f(.  =  lti;  If)  f(.  =  (gl  f(.  =  72: 
lhlfe=  i  12:  lit  f(.  =  128.  iSeetioii  14.81 


;  2.88 

i 


Figure  82.  Ono-Oila^cf.hanni'l  Filtered  Portrait.  fr  —  Outre  Frequency  of  Filter:  la  I  Original 
Portrait:  till  f(.  =  I:  lei  f(.  =  2:  =  1;  lei  f,.  =  8;  Ifl  f(.  =  It);  (K|f,.  =  .{2;  thl  f(.  = 

iSeetion  14. .'$1 


(a) 


(b) 


(c) 


f  igore  83.  Two-Octave  (Channel  Filtered  Portrait.  f(.  =  Centre  Frequency  of  Filter:  (al  Original 

Portrait:  (hi  f,.  =  I;  Id  f,.  =  2:  (dl  f,.  =  4;  lei  f,.  =  8;  lrl  f(.  -  It.;  IK|  f(.  =  32;  Ihl  f,.  =  (4; 
lit  f,-  =  128.  (Section  14.31 
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SNELLEN 
LINE  NUMBER 


SIZE  OF 
LETTER 


SPATIAL  FREQUENCIES  OF 
LETTERS  (c/d) 


min 

c/d 

1.0 

1.5 

2.0 

2.5 

6/60 

20/200 

50.0 

1.2 

0.6 

1.2 

1.8 

2.8 

3.0 

6/36 

20/120 

30.0 

2.0 

1.0 

2.0 

3.0 

4.0 

5.0 

6/24 

20/80 

20.0 

3.0 

4.5 

3.0 

4.5 

6.0 

7.5 

6/18 

20/60 

15.0 

4.0 

2.0 

4.0 

6.0 

8.0 

10.0 

6/12 

20/40 

10.0 

6.0 

3.0 

6.0 

9.0 

12.0 

15.0 

6/9 

20/30 

7.5 

8.0 

4.0 

8.0 

12.0 

16.0 

20.0 

6/6 

20/20 

5.0 

12.0 

6.0 

12.0 

18.0 

24.0 

30.0 

6/5 

20/15 

4.2 

14.4 

7.2 

14.4 

21.6 

28.8 

36.0 

6/3 

20/10 

2.5 

24.0 

12.0 

24  0 

36.0 

48.0 

60.0 

f1  inure  K4.  Table  of  First  2.5  Cycles  of  Letters  Associated  with  Each  Snellen  Line 
Number.  (Section  10.2 1 
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(6) 


(8) 


F 


(7) 


Figure 8">.  Synthesis  of  Snellen  Letters:  la)  Letter  K:  lb)  letter  L; 
1 1 1  lxm-l’ass  l.fic/ljetlorl  Filtered  letters; 

(21  Low-Pass  ( I  .Oe/Letterl  Filtered  Letters; 

131  Low-Pass  1 1  e/Letterl  Filtered  Letters: 

(41  Low-Pass  (2.0  e/Letterl  Filtered  letters; 

(.'»)  ls>w-Pass  (2..)  e/ letter  I  Filtered  Letters; 
ltd  Low-Pass  1.3.0  e/Letterl  Filtered  letters; 

IT)  Band-Pass  1 1  -  3  r/I/etterl  Filtered  Letters 

(8)  Band-Pass  (2  -  3  c/I,etterl  Filtered  Letters. 

(Seetion  1(>.3I 


V  O  A 

T  X  A  L 

O  A  N  V  Z 

HZNVTUE 

NOHXE  Z  A  U 

Figure  8<>.  Snellen  Letter  Acuity  Chart  Used  in 
experiments.  (Section  17.21 
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Figure  87.  Table  of  Contrast  and  Contrast  Sensitivity  Required  for  Detection  and  Identification  of  Snellen  Letters. 
(Section  17.31 


%  CONTRAST  REQUIRED  FOR  DETECTION 
AND  IDENTIFICATION  OR  SNELLEN  LETTERS 


SNELLEN  LINE  NUMBER 


Figure  88.  The  Percent  Contrast  Required  for  the  Detection  and  Identification  of 
Snellen  Letters  for  Each  Snellen  Line.  (Section  17.81 


2W> 


CONTRAST  SENSITIVITY  OF  DETECTION  AND  IDENTIFICATION 
OF  SNELLEN  LETTERS 


(a) 


SPAT  l  At  rRF  .}UrV.'Y  (  CYCLES  /  DEGREE  ) 


30  30  10  10  7  5  5  4  3 

LETtpr  SIZE  (MIN) 

i - 1 - 1 - 1 - 1 - 1 - 1 _ i 

6/60  6/36  5/?46'lB  6/i26/9  6/6  6/5 

SNFlIFN  LINE  NUMBER 


2  1  5 


Figure  84.  Contrast  Sensitivity  Required  for  the  Detection  and 

Identification  of  Snellen  Letters  for  Each  Snellen  Line:  la  I  Log 
Contrast  Sensitivity  -  Log  Spatial  Frequency;  (bl  Log 
Contrast  Sensitivity  -  Linear  Spatial  Frequency.  (Section  17.31 
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Figure  8'».  Contrast  Sensitivity  Required  for  the  Detection  and  Identification  of  Snellen  Letters  for  Each  Snell.  Line: 

lal  Log  Contrast  Sensitivity  -  Log  Spatial  Frequency:  Ibl  Log  Contrast  Sensitivity  -  Linear  Spatial  Frequency. 
ISection  17.31  Icont.l 


RATIO  OF  CONTRAST  REQUIRED  FOR  IDENTIFICATION 
VERSUS  DETECTION  OF  SNELLEN  LETTERS 


CONTRAST  SENSITIVITY 


SPATIAL  FREQUENCY 
(  cycles  /degree) 


Figure  9 1 .  Contrast  Sensitivity  Function  of  Subject  A.G.  (Section  1 7.3 1 
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SNELLEN 

LINE 

NUMBER 


SPATIAL  FREQUENCIES 
OF  LETTERS  REQUIRED  FOR 
IDENTIFICATION 


(c/d) 


CONTRAST  SENSITIVITY 
REQUIRED  FOR 
IDENTIFICATION 


6/60 

20/200 

6/36 

20/120 

6/24 

20/80 

6/18 

20/60 

6/12 

20/40 

6/9 

20/30 

6/6 

20/20 

6/5 

20/15 

1.5 

2.0 

1.8 

2.4 

3.0 

4.0 

4.5 

6.0 

6.0 

8.0 

9.0 

12.0 

12.0 

16.0 

18.0 

24.0 

21.6 

28.8 

2.5 

-2  S.l 

3.0 

32.3 

5.0 

30.3 

7.5 

24.4 

10.0 

21.7 

15.0 

13.3 

20.0 

8.8 

30.0 

6.1 

36.0 

2.7 

AVE 

+2  S.E 

39.9 

45.5 

35.6 

43.5 

27.7 

30.3 

25.1 

27.0 

16.6 

21.3 

10.8 

13.9 

7.1 

8.3 

3.6 

4.6 

Figure  92.  Table  of  Cycles  and  Associated  Contrast  Sensitivity  Required  for  the  Identification  of 
Snellen  Letters.  (Section  18.1,  18.21 


MEASURED  PREDICTED  SNELLEN 


MEASURED  SNELLEN 

CONTRAST 

LETTER  ACUITY 

SUBJECT 

LETTER  ACUITY 

SENSITIVITY 

AVE 

±2  S.E. 

JP 

6/ 5 (R) 

3.2 

6/ 6 (R) 

6/5(R) 

* 

6/9(L) 

8.9 

6/12 (L) 

6/ 9 (L) 

* 

MB 

6/ 4 (R) 

3.6 

6/ 4 (R) 

6/ 4 (R) 

* 

2/60(L) 

51.2 

6/ 60 (L) 

6/60(L) 

AH 

6/5 (R) 

1.5 

6/ 9 (R) 

6/9(R) 

6/ 36 (L) 

10.2 

6/60(L) 

6/60 (L) 

t 

AH 

6/ 5 (R) 

6.2 

6/5 (R) 

6/5 (R) 

* 

6/18 (L) 

24.1 

6/18 (L) 

6/18 (L) 

k 

RH 

6/9(R) 

15.2 

6/9(R) 

6/9 (R) 

* 

6/ 5 (L) 

6.4 

6/5 (L) 

6/ 5 (L) 

* 

MD 

6/ 6  (R) 

2.5 

6/12 (R) 

6/9 (R) 

t 

6/ 6 (L) 

1.3 

6/18 (L) 

6/ 18 (L) 

JJ 

6/6 (R) 

2.3 

6/9(R) 

6/9 (R) 

t 

6/9(L) 

4.8 

6/24 (L) 

6/18(L) 

SS 

6/ 6 (R) 

8.9 

6/6 (R) 

6/ 6 (R) 

* 

6/ 6 (L) 

3.6 

6/ 9  (L) 

6/9 (L) 

t 

BM 

6/6 (R) 

4.5 

6/9 (R) 

6/9 (R) 

t 

6/6 (L) 

1.5 

6/18(L) 

6/18 (L) 

HR 

6/6(R) 

3.8 

6/ 9 (R) 

6/9 (R) 

t 

6/12(L) 

5.1 

6/18 (L) 

6/12 (L) 

* 

CH 

6/ 6 (R) 

15.2 

6/ 6 (R) 

6/ 6 (R) 

•k 

6/ 6 (L) 

8.9 

6/ 6 (L) 

6/ 6 (L) 

k 

*  =  AGREEMENT  BETWEEN  MEASURED  AND 
PREDICTED  SNELLEN  LETTER  ACUITY 

t  =  AGREEMENT  OFF  BY  ONE  SNELLEN  LINE 


Figure  ‘XL  Table  of  Predicted  Snellen  Letter  Acuity  from  the  Contrast  Sensitivity  of  10  Subjects. 
(Section  18.21 
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6/4  S/g  6/^2  6/24 

PREDICTED  SNELLEN 
LETTER  ACUITY 


6/60 


Figure  'M.  Measured  Versus  Predicted  Snellen  Letter  Acuity.  (Section  18.21 


100 


1000 1 


~  100 


Subject  J  P 

Visual  6/  R 

'5 

Acuity  6/  L 
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Right  eye  • 
Left  eye  O 


•so  y 
*15  5 

+  10  LU 


1  10 
SPATIAL  FREQUENCY 
(  eye  les  /  degree) 


Figure  (>7.  Contrast  Sensitivity  Functions  and  Difference  Between  Fives  Curve 

Amblyope  J.P.  l/oss  at  All  Frequencies.  Stationary  Sine- Wave  Gratings. 
iSection  A -.1 . 2 1 


BETWEEN  EYES 
(  decibels) 


1000 


Subject  M  B 
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Visual  6/4  R 
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Left  eye  O 
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O 


i _ i _ i _ J-5 

1  1  10  100 

SPATIAL  FREQUENCY 
(cycles/degree) 


t  'ML  ( .ontrast  Sensitivity  Functions  and  Difference  Between  Kyes 
Curve:  Amblyope  M.B.  D>ss  at  High  f  requencies.  Stationary 
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Figiirf  W.  Contrast  Sensitivity  Functions  and  Difference  Between  Fves  Curve: 

Amhlyope  A.H.  N>ssat  Hitfh  Frequencies  and  Selective  Mid- Frequency 
lx>ss.  Stationary  Sine-Wave  Gratings.  (Section  A-3.4. 1 1 
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Figure  HH).  Contrast  Sensitivity  Functions  and  Difference  Between  Kves  Curve:  Amblyope 
A.H.  Ixtss  at  High  Frequencies  and  Selective  Mid-Frequency  Loss.  Stationary 
Sine-Wave  Gratings.  (Section  A-3.4. 1 1 
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Figure  101 .  Average  Contrast  Sensitivity  Functions  and  Difference  Between  Fives 
Curve  of  F  igures  00  and  1 00:  Stationary  Sine-\X  ave  Gratings. 
(Section  A-.'i.  f.  1 1 


281 


SPATIAL  FREQUENCY 
( cycles  /  degree) 


Figure  1 02.  Decrease  in  Difference  Between  Eyes  Curve  for  the  First  Two  Test 
Sessions:  Amblyope  A.H.  Stationary  Sine-Wave  Gratings. 
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Figure  103.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Effect  of  Abnormal  Retinal  Correspondence  (ARC).  Stationary 
Sine-Wave  Gratings.  (Section  A-2.4.1) 
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Figure  105.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 

Ambiyope  A.H.  Loss  at  High  Frequencies  and  Selective  Mid-Frequency 
Loss.  Moving  Sine-Wave  Gratings.  (Section  A-4.2) 


285 


CONTRAST  SENSITIVITY 


SPATIAL  FREQUENCY 
(cycles  /degree) 


Figure  106.  Average  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes 

Curve  of  Figures  104.  105.  Moving  Sine- Wave  Gratings.  (Section  A-3.4.21 
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Figure  107.  Decrease  in  Difference  Between  Eyes  Curve:  Amblyope  A.H. 
Moving  Sine- Wave  Grating.  (Section  A-3.4.2) 
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Figure  108.  Average  from  Three  Seta  of  Contraat  Sensitivity  Functions  and 
Difference  Between  Eyes  Curves:  Amblyope  A.H.  Stationary 
Square-Wave  Gratings.  (Section  A-3.4.3) 
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Figure  109.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 

Amblyope  A.H.  Stationary  Sine-  and  Square-Wave  Gratings  from  One 
Test  Session.  (Section  A-3.4.31 
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Figure  111.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Amblyope  A.H.  Stationary  Sine-Wave  Gratings.  (Section  A-3.5) 
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Figure  1 12.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Amblyope  A.H.  Stationary  Sine-Wave  Gratings.  (Section  A-3.5) 
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Figure  1 13.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Amblyope  A.H.  Stationary  Sine-Wave  Gratings.  (Section  A-3.51 
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Figure  1 14.  Contrast  Sensitivity  Versus  Time:  Amblyope  A.H.  (Section  A-3.51 
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Figure  1 20.  Difference  Between  Eyes  Curves  for  each  Contrast  Match  for  Two 
Separate  Test  Sessions.  (Figures  ]  10a,  bl  and  Their  Average  (Figure 
HOcI:  Amblyope  A.H.  (Section  A-4.21 
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Figure  121 .  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Multiple  Sclerosis  C.H.  Mid-Frequency  Loss.  (Section  A-5.21 
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Figure  122.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Multiple  Sclerosis  S.S.  All  Frequency  Loss.  (Section  A-5.21 
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Figure  123.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Multiple  Sclerosis  H.R.  All  Frequency  Loss.  (Section  A-5.21 
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Figure  125.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Multiple  Sclerosis  M.D.  Mid-Frequency  Loss.  (Section  A-5.21 
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Figure  126.  Contrast  Sensitivity  Functions  and  Difference  Between  Eyes  Curve: 
Multiple  Sclerosis  B.M.  Predominant  High-Frequency  Loss. 
(Section  A-5.2) 
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APPENDIX  A 


INVESTIGATIONS  INTO  THE  FILTERING 
CHARACTERISTICS  OF  ABNORMAL  VISION 


CHAPTER  A-l 


THE  FILTERING  CHARACTERISTICS  OF  ABNORMAL  VISION 


A- 1.1  Why  Investigate  Abnormal  Vision? 

Chapters  9-14  were  concerned  with  the  effects  of  the  filtering  characteristics  of  normal  vision  on  the  perception  of  many 
objects.  In  these  appended  chapters,  the  filtering  characteristics  of  abnormal  vision  are  examined.  The  motivation  for 
this  particular  research  is  two-fold.  First,  an  understanding  of  visual  perception  in  an  abnormal  system  may  lead  to  a 
better  understanding  of  how  mechanisms  of  the  normal  visual  system  function.  Second,  a  subject  with  a  particular  kind 
of  abnormal  vision  could  be  used  to  test  certain  hypotheses  based  upon  research  in  normal  vision,  as  was  discussed  in 
Section  15.7. 

A- 1.2  Amblyopia 

If  the  neurophysiological  filtering  properties  of  abnormal  vision  are  to  be  investigated,  then  a  condition  should  be 
chosen  that  is  not  optical  in  origin.  Therefore,  the  abnormal  visual  conditions  investigated  were  amblyopia  and 
multiple  sclerosis. 

Initial  investigations  began  with  a  visual  disorder  that  affects  about  four  percent  of  the  population:  amblyopia.  The 
brief  information  about  amblyopia  presented  in  the  following  section  has  been  gleaned  from  Duke-Elder  (19491, 

Borish  (1970),  and  Lyle  and  Wybar  ( 19701.  In  general,  amblyopia  is  described  as  a  dimness  of  form  vision  that  cannot 
be  corrected  by  optical  means.  The  most  common  cause  of  amblyopia  is  the  lack  of  use  of  one  eye  early  in  life  due  to 
strabismus  (squint).  One  eye  becomes  dominant  and  is  used  almost  exclusively,  to  the  detriment  of  the  other  eye. 

Von  Noorden  ( 1967)  provided  a  recent  classification  of  different  types  of  amblyopia.  In  addition  to  strabismic 
amblyopia,  there  is  also  amblyopia  ex  anopsia  (amblyopia  of  disuse),  anisometropic  amblyopia  (amblyopia  due  to  a 
difference  in  the  refractive  power  between  the  two  eyes),  aniseikonia  (unequal  size  of  images  of  each  eye),  congenital 
amblyopia  (organic  malfunction  which  includes  receptor  amblyopia  secondary  to  nystagmus  and  achromatopia).  and. 
finally,  ametropic  amblyopia  (amblyopia  due  to  poor  refractive  power  of  the  eye).  Although  seemingly  similar,  Von 
Noorden  distinguishes  ametropic  amblyopia  from  anisometropic  amblyopia  by  noting  that  the  interference  in  the  visual 
system  between  unequal  images  that  occurs  in  anisometropic  or  l  -  .abismic  amblyopia  never  occurs  in  ametropic 
amblyopia. 

An  interesting  comment  on  amblyopia  was  made  by  Duke-Elder  (1949):  “Every  child  may  be  said  to  be  born 
bilaterally  amblyopic,  and  it  is  only  through  the  constant  facilitation  of  reflexes  cumulatively  sustained  by  the  reward  of 
clear  vision  that  epicritic  vision  is  attained."  Thus,  normal  development  of  the  visual  system  may  be  needed  to 
overcome  an  initial  amblyopic  state. 

If  an  abnormal  condition  persists  and  is  not  corrected,  amblyopia  may  result.  This  can  cause  two  problems:  confusion 
and  diplopia  (double  vision).  Confusion  results  from  the  spatial  overlap  of  two  different  images,  which  occurs  because 
corresponding  retinal  points  are  stimulated  by  two  different  images.  Diplopia  results  from  having  the  same  images 
appearing  side  by  side.  This  occurs  because  noncorresponding  retinal  points  are  stimulated  by  the  same  image. 

The  visual  system  uses  three  general  adaptive  mechanisms  to  inhibit  or  suppress  vision  in  order  to  abolish  or  at  least 
diminish  confusion  and  diplopia:  facultative  inhibition,  obligatory  inhibition,  and  eccentric  retinal  fixation. 

Faculative  inhibition  only  occurs  in  the  squinting  eye  and  disappears  when  it  resumes  fixation.  Obligatory  inhibition 
is  a  sequel  to  facultative  inhibition  and  persists  even  when  the  squinting  eye  fixates.  Eccentric  retinal  fixation  may 
follow  obligatory  inhibition  and  refers  to  the  nonfoveal  receptors  that  assume  a  foveal  form  of  projection  as  a  point  of 
reference  for  other  retinal  receptors.  The  change  in  fixation  point  is  not  too  rewarding  because  the  foveal  type  of 
projection  cannot  equal  the  foveal  type  of  acuity.  Eccentric  retinal  fixation  is  usually  preceded  by 
abnormal  retinal  correspondence. 


A-2 


Abnormal  retinal  correspondence  I  ARC)  occurs  when  the  projection  of  the  retinal  receptors  of  the  squinting  eye  are 
centered  on  a  retinal  area  other  than  the  fovea  and  assume  a  foveal  type  of  projection.  It  differs  from  eccentric  retinal 
fixation  by  not  being  maintained  as  the  fixation  point  when  the  squinting  eye  is  used  alone.  In  uncomplicated  ARC  the 
inhibition  disappears  on  covering  the  nonsquinting  eye  and  may  be  regarded  as  facultative,  and  a  reasonable  level  of 
vision  is  maintained.  However,  in  some  cases  of  ARC  the  abnormal  state  is  obligatory  and  remains  after  the 
nonsquinting  eye  is  covered.  In  such  cases,  ARC  is  associated  with  eccentric  retinal  fixation  and  visual  acuity  is 
usually  poor. 

A- 1.2.1  Early  Psychophysical  Studies  of  Amblyopia 

The  psychophysics  of  amblyopia  has  been  extensively  investigated  over  the  last  35  years  (see  review  by  Burian,  1%7I. 
Amblyopes  have  normal  light  and  dark  adaptation  and  can  localize  points  of  light  (Vogt.  1939;  Wald  and  Burian, 

19441.  Miller  ( 19541  investigated  brightness  discrimination  and  summation  in  amblyopia.  He  found  that  the  differential 
luminance  threshold  under  photopic  conditions  increased  as  the  width  of  a  stimulus  line  increased.  However, 
decreasing  the  width  of  the  line  to  less  than  4  to  6  minutes  in  the  normal  eye  and  10  minutes  in  the  amblyopic  eye  did 
not  affect  threshold.  Miller  concluded,  in  agreement  with  the  earlier  results  of  Wald  and  Burian  (19441,  that  the  fovea 
of  the  amblyope  behaved  like  the  periphery  of  a  dark  adapted  normal  retina.  Similar  results  were  found  by  Grosvenor 
( 19541  and  Meur  ( 1964). 

In  general,  the  spatial  summation  properties  of  the  amblyopic  fovea  seem  similar  to  those  of  the  normal  peripheral 
retina.  The  most  accepted  theory  for  reduced  photopic  functioning  in  the  amblyopic  fovea  was  put  forward  by  Miller 
( 19551:  reduced  lateral  inhibition  between  foveal  cones.  Miller’s  theory  is  attractive  because  it  seems  to  explain  certain 
clinical  phenomena,  for  example,  the  "crowding  effect"  in  which  acuity  is  higher  for  single  letters  than  for  lines  of 
letters.  Presumably,  the  reduced  lateral  inhibition  between  the  foveal  cones  causes  the  letters  to  be  smeared  together 
and  interferes  with  their  legibility  when  formed  in  a  line. 

Although  the  crowding  effect  is  usually  associated  with  amblyopia,  it  was  associated  with  visual  acuity  by  Flom. 

W  eymouth,  and  Kahneman  (1963 1.  They  found  that  the  effect  of  contour  interaction  with  visual  resolution  is  similar  in 
both  normal  and  amblyopic  subjects. 

All  the  evidence  to  date  suggests  that  amblyopia  results  from  cortical  inhibition.  For  example,  persons  born  with  a  high 
degree  of  astigmatism  still  retain  poor  vision  in  the  astigmatic  meridian  even  with  correcting  lenses  (Martin.  1890; 
Luedde,  1922;  Mitchell.  Freeman,  Millodot,  and  Haegerstrom,  1973).  These  findings  strongly  suggest  that 
cortical  mechanisms  are  involved  in  amblyopia  since  the  cortex  is  the  first  site  of  orientation  receptive  fields. 
Furthermore,  since  the  cortex  contains  binocularly  excited  neurons,  it  seems  reasonable  from  an  economy  of  effort 
viewpoint  that  the  cortex  should  be  involved  in  the  suppression  of  diplopia. 

A- 1.2. 2  Recent  Studies  of  Amblyopia 

Much  general  information  about  amblyopia  has  been  found  using  the  conventional  psychophysical  paradigms. 
However,  the  use  of  contrast  sensitivity  measurements  is  a  more  systematic  approach  which  yields  more  quantitative 
information  than  could  have  been  otherwise  obtained. 

Gstalder  and  Green  ( 1971 )  measured  the  acuity  of  amblyopes  using  laser  interferometric  gratings,  sine-  and 
square-wave  gratings,  and  Snellen  letters.  They  found  that  acuity  of  amblyopes  was  greatest  with  gratings  and 
greater  with  laser  interferometric  gratings  than  with  Snellen  letters.  Although  they  suggested  that  the  laser  generated 
gratings  resulted  in  the  greatest  acuity,  luminance  and  criterion  differences  that  existed  in  their  experiments  make  that 
conclusion  questionable.  They  also  claimed  that  the  measured  contrast  sensitivity  was  normal  for  low  and  medium 
spatial  frequencies  between  1  and  6  c/d  for  the  amblyopic  eye  compared  to  the  normal  eye  in  the  two  subjects. 

However,  the  contrast  sensitivity  function  shown  in  Figure  1  of  their  paper  shows  similar  values  only  at  1  and  2  c/d. 
Furthermore,  one  measurement  at  1  c/d  does  not  constitute  enough  data  with  which  to  generalize  about  contrast 
sensitivity  at  the  low  spatial  frequencies. 


Hess  ( 19741  in  an  extensive  study  using  contrast  sensitivity  measurements  found  two  general  classes  of  strabismic 


amblyopia.  One  class  showed  losses  in  contrast  sensitivity  predominantly  over  the  high  spatial  frequencies.  The  second 
class  showed  losses  in  contrast  sensitivity  over  the  whole  range  of  spatial  frequencies.  In  their  study  of  several  clinical 
visual  disorders,  Itoi,  Kato,  Sugimachi,  Kamamura,  Ohzu,  and  Kawara  (19751  showed  selective  spatial  frequency 
losses  at  middle  spatial  frequencies  in  three  strabismic  amblyopes  and  two  ametropic  amblyopes.  Unfortunately,  they 
did  not  measure  sensitivity  below  2  c/d.  Most  recently,  Levi  and  Harwerth  ( 19771  investigated  both  spatial  and 
temporal  interactions  using  gratings  in  cases  of  anisometropic  and  strabismic  amblyopia.  They  found  a  decrease  in 
sensitivity  at  all  spatial  frequencies  for  both  long  (500  msec)  and  short  (50  msec)  grating  presentations  for  the  amblyopic 
eye.  However,  the  contrast  sensitivity  for  the  low  spatial  frequencies  was  about  the  same  in  both  the  normal  and  the 
amblyopic  eye  if  the  grating  was  flickered  at  10  Hz.  Such  results  suggested  that  there  are  differences  between  the 
perception  of  static  and  dynamic  patterns  in  the  amblyopic  eye. 

Such  selective  spatial  frequency  losses  in  amblyopia  are  difficult  to  explain  without  a  concept  of  channels  tuned  to 
limited  ranges  of  spatial  frequency  in  the  visual  system.  With  this  limited  background,  the  spatial  stimuli  and 
experimental  procedures  used  to  determine  the  filtering  characteristics  of  some  amblyopes  are  examined  next. 
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CHAPTER  A-2 


THE  GENERATION  OF  STIMULI  AND  EXPERIMENTAL  PROCEDURES 


A-2. 1  The  Generation  of  Spatial  Stimuli 

The  visual  stimuli  were  displayed  on  the  lace  of  a  standard  television  cathode  ray  tube  (CRTl  using  techniques 
developed  by  Schade  119561  and  modified  by  Robson  119661.  The  display  consisted  of  a  raster  of  about  1000  lines 
achieved  by  applying  a  250  Hz  sawtooth  waveform  and  a  150  Hz  triangular  waveform  to  the  x-  and  y-axis  of  the  CRT, 
respectively.  One  frame,  the  required  time  for  the  beam  to  form  the  raster  along  the  x-axis,  took  about  4  msec.  Both 
the  frame  frequency  —  about  five  times  higher  than  the  highest  flicker  frequency  detectable  by  the  visual  system — and  the 
high  density  of  the  lines  were  unresolvable  for  normal  viewing  distance. 

The  mean  luminance  of  the  raster  was  provided  by  a  steady  voltage  into  the  z-axis  of  the  CRT  and  was  kept  at  about  40 
foot-lamberts  which  is  in  the  photopic  range  of  vision.  A  P31  phosphor  provided  a  short  persistence  light  having  a 
broad  wavelength-spectrum  with  a  maximum  in  the  blue-green  spectral  region.  The  unmodulated  raster  appeared 
to  be  desaturated  green  that  was  spatially  and  temporally  uniform. 

A-2. 1.1  Contrast 

The  raster  was  modulated  by  applying  a  modulation  signal  to  the  z-axis  from  a  standard  500-10  kHz  oscillator. 

Sinusoidal  modulation  produced  a  sinusoidal  grating  that  did  not  change  the  mean  luminance  of  the  screen.  As 
previously  discussed,  the  depth  of  modulation  about  the  mean  luminance  determined  the  contrast. 

Over  the  range  of  contrasts  studied,  the  luminance  of  the  screen  {measured  by  a  linear  photocell)  varied  linearly  with 
the  modulation  voltage. 

A  .  2. 1.2  The  Stimuli 

The  gratings  used  in  all  the  experiments  were  vertically  oriented  with  luminance  varying  along  the  horizontal  axis  about 
a  constant  mean  luminance. 

A-2. 1.2. 1  Stationary  Gratings 

A  stationary  grating  was  achieved  by  synchronizing  the  frame  frequency  of  the  CRT  with  the  frequency  of  the  z-axis 
modulation  signal.  The  grating  can  be  described  by 

Llxl  =  L(!+m  •  sin2irfxx) 

where  L  is  the  mean  luminance,  m  is  the  contrast,  and  f  is  the  spatial  frequency.  As  a  reminder,  spatial  frequency  is 
defined  as  the  number  of  cycles  of  the  grating  per  degree  of  visual  angle  (c/d ). 

A-2. 1.2. 2  Moving  Gratings 

Gratings  can  be  made  to  appear  to  move  along  the  horizontal  axis  of  the  CRT.  A  moving  grating  can  be  described  by 
Llx,  tl  =  L[1  +  m  •  sin  27T(fx  +  Vt)  ] 

where  V  is  the  velocity  of  the  movement  in  degrees  of  visual  angle  per  sec  (d/sec  I.  Since  V  =  ft/fx.  where  ft  is  the 
temporal  frequency  in  hertz,  the  moving  grating  can  be  described  by 

Llx,  tl  =  L[l  +  m  •  sin  2  it  (fxx  +  f|tl] 

Using  a  trigonometric  identity  yields 

sin  2 7T(fxx  +ftt)  =  sin  2TTfxx  •  cos  2irftt  +  cos  2trfxx  •  sin  2trftt 

That  is,  a  moving  grating  is  the  sum  of  two  sinusoidally  modulated  gratings  that  are  90  degrees  out  of  phase  with 
respect  to  both  space  and  time. 
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A-2. 1.2.3  Split-Screen  Gratings 


I'he  contrast  matching  experiments  require  the  independent  control  of  spatial  frequency  and  contrast  of  two  sinusoidal 
grating  displays.  In  order  to  avoid  any  problems  associated  with  matching  two  different  display  systems,  a  split-screen 
technique  was  used  to  prov  ide  the  two  independent  displays  using  the  same  CRT.  The  split  screen  was  made  by  using 
the  trigger  signal  from  the  CRT  that  provided  the  frame  rate  to  synchronize  two  oscillators  and  trigger  a  pulse 
generator.  The  pulse  generator  provided  a  signal  at  twice  the  frequency  of  the  frame  rate  that  was  used  to  drive  an 
electronic  switch.  The  electronic  switch  alternately  connected  the  output  of  each  oscillator  to  the  z-axis  of  the  CRT 
which,  in  turn,  created  two  half-frames  on  the  CRT.  Although  two  gratings  were  adjacent  to  one  another  on  the  same 
screen,  the  spatial  frequency  and  contrast  of  each  half-frame  could  be  controlled  independently. 

The  luminance  distribution  of  the  split  screen  can  be  described  by 
Ll  x  I  =  Ll  1  +  m  |  •  sin  f x .  x  I  for  x  5  0,  x/2 

=  Lll  +mi  •  sinfx.,x!  forx  »x/2,  x 

A  white  2  cm  wide  strip  of  cardboard  was  placed  over  the  CRT  to  separate  the  two  fields  and  prevent  the  subject  from 
using  any  information  from  the  portions  of  the  gratings  that  were  immediately  adjacent  to  one  another  in  the 
experiments. 

A-2. 1 .3  Determining  Contrast 

Contrast  of  the  gratings  was  determined  by  passing  the  output  of  the  oscillator  through  a  potentiometer  that  attenuated 
the  grating  39  dB  in  0.025  log  unit  steps.  The  base  contrast  was  determined  with  the  attenuator  set  at  zero.  A  neutral 
density  filter  of  known  value  was  placed  over  a  square-wave  grating  that  was  displayed  on  the  CRT.  The  oscillator 
output  was  then  adjusted  to  make  the  light  and  dark  bars  appear  to  be  of  equal  brightness.  A  neutral  density  of  0.3  was 
used  to  create  a  base  contrast  of  33  percent.  Higher  values  of  contrast  were  determined  by  a  decade  attenuator  in  series 
with  the  potentiometer  and  use  of  a  nonzero  setting  for  the  base  contrast.  In  this  way.  higher  contrasts  could  he 
obtained  by  decreasing  the  decade  attenuator. 

The  logarithmic  potentiometer  was  mechanically  linked  to  a  second  potentiometer  which  provided  a  linear  output 
voltage  in  direct  proportion  to  the  logarithmic  potentiometer.  The  output  voltage  went  to  the  analogue/digital 
converter  of  a  PDP  8/1  digital  computer  for  the  calculation  of  contrast  sensitivity. 

The  display  screen  was  surrounded  by  a  white  card  that  was  illuminated  by  a  projector  to  about  the  same  mean 
luminance  as  the  CRT.  This  was  done  to  reduce  the  fatigue  of  the  observer  that  can  occur  when  he  makes  many 
threshold  measurements  over  a  visual  area  that  has  large  differences  in  contrast.  Furthermore,  it  insures  a  constant 
level  of  adaptation  of  the  eye  as  it  wanders  over  the  whole  display. 

A-2-2.  Measuring  Contrast  Sensitivity 

The  experimental  condition  for  the  observers  during  the  measurement  of  contrast  sensitivity  was  as  follows.  The  subject 
randomly  scanned  a  CRT  across  which  a  sine-wave  grating  of  zero  contrast  was  presented.  A  blank  screen  of  uniform 
luminance  was  seen.  The  contrast  of  the  grating  was  increased  until  it  could  only  just  be  seen.  The  lowest  contrast  at 
which  the  grating  was  seen  is  called  the  contrast  threshold.  The  reciprocal  of  the  contrast  threshold  is  called  the  contrast 
sensitivity. 

rrhe  contrast  of  the  grating  was  controlled  by  means  of  a  potentiometer,  as  previously  discussed.  When  the  subject  set 
the  contrast  threshold,  he  pressed  a  microswitch  which  enabled  the  computer  to  read  the  voltage  across  the 
potentiometer  and  to  compute  the  contrast  of  the  grating.  At  least  ten  consecutive  measurements  were  made  at  each 
spatial  frequency  and  the  mean  was  computed  and  used  to  compute  the  value  of  contrast  sensitivity.  The  viewing 
distance  was  57  cm  for  the  measurements  of  the  spatial  frequencies  at  and  below  2  c/d.  The  viewing  distance  was  229 
cm  for  the  measurements  of  high  spatial  frequencies  above  2  c/d.  Appropriate  lenses  were  placed  before  the  eye  being 
tested  to  provide  in-focus  gratings  with  minimum  accommodative  effort  by  the  subject.  The  square  grating  field 
subtended  24.5  degrees  by  19.3  degrees  at  57  cm.  and  6. 1  degrees  by  4.8  degrees  at  229  cm. 


This  technique  for  obtaining  threshold  measurements  is  called  the  method  of  adjustment.  It  is  a  simple  and  popular 
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method  for  measuring  contrast  thresholds.  But  it  is  open  to  criticism,  mainly  because  it  is  subjective  and  subjects  can 
adopt  different  criteria  for  the  amount  of  a  grating  that  is  seen  at  threshold.  Other  alternative  techniques,  such  as  the 
method  of  constant  stimuli,  requires  a  considerable  amount  of  time  for  data  collection.  The  time  required  for  such 
measurements  and  the  tedium  involved  would  be  unrealistic  for  the  number  of  subjects  and  physical  conditions  (e.g.. 
multiple  sclerosis  I  that  were  used  in  these  studies. 

Another  reason  the  method  of  adjustment  was  chosen  is  that  the  vast  majority  of  data  was  to  be  used  to  compare  the 
performance  of  one  eye  against  the  other  eye  in  the  same  subject.  Thus,  small  individual  differences  in  the  criterion  for 
threshold  would  not  be  important  so  long  as  the  criterion  of  the  subject  remained  the  same  for  each  eye  during  each  set 
of  measurements. 

The  ascending  method  of  adjustment  was  chosen.  That  is,  the  contrast  threshold  was  obtained  by  increasing  the 
contrast  of  the  grating  until  it  was  seen  rather  than  decreasing  the  contrast  of  a  high  contrast  grating  until  it  was  not 
seen.  Inexperienced  subjects  had  no  difficulty  with  this  method  which  avoids  any  effects  of  adaptation  of  gratings  of 
higher  contrast  that  could  result  in  a  higher  threshold  (Blakemore  and  Campbell,  1%9I. 

A-2.3  Measuring  a  Contrast  Match 

A  split-screen  CRT  with  one  side  containing  a  vertical  grating  of  one  spatial  frequency  and  contrast  and  the  other  side 
a  vertical  grating  of  another  contrast  and  sometimes  a  different  spatial  frequency  was  used  for  measuring  a  contrast 
match.  The  subject  adjusted  the  contrast  of  one  of  the  gratings  (the  variable!  until  it  appeared  to  be  the  same  as  that  of 
the  other  grating  (the  standard).  This  procedure  is  called  contrast  matching. 

The  measurements  reported  here  were  made  in  a  manner  partially  similar  to  that  of  Georgeson  and  Sullivan  ( 19751 
in  an  attempt  to  replicate  their  results.  The  major  difference  using  this  method  was  the  selection  of  a  standard  grating 
of  2  c/d  rather  than  5  c/d  because  previous  contrast  sensitivity  measurements  using  this  apparatus  were  maximum  at 
about  2  c/d.  Thus,  the  contrast  matches  were  made  between  a  standard  sinusoidal  grating  of  2  c/d  and  variable 
gratings  whose  spatial  frequencies  ranged  from  0.2  to  25  c/d.  The  contrast  of  the  standard  grating  ranged  from  0.03 
percent  to  66  percent  in  6  db  steps.  As  with  the  measurement  of  contrast  sensitivity,  two  viewing  distances  were  used: 

57  cm  for  spatial  frequencies  2  c/d  and  below;  229  cm  for  spatial  frequencies  2  c/d  and  above.  Appropriate  lenses  were 
placed  before  the  eye  to  create  in-focus  gratings  with  near  zero  accommodative  effort  by  the  subject.  At  57  cm  each  field 
subtended  12  degrees  by  19.3  degrees  separated  by  a  1  degree  wide  strip  of  cardboard.  A  2.5  degree  strip  separated  the 
3  by  4.8  degree  fields  when  viewed  from  229  cm.  Monocular  viewing  was  used  and  the  observer  was  allowed  as  much 
time  as  desired  to  look  back  and  forth  at  the  two  gratings. 

The  contrast  of  the  standard  grating  was  set  by  the  experimenter.  The  contrast  of  the  variable  grating  was  adjuster!  by 
the  observer  with  a  logarithmic  potentiometer  for  a  contrast  match.  In  general,  five  contrast  matches  for  each  condition 
were  obtained  at  each  session.  The  contrast  of  the  standard  grating  was  changed  in  either  ascending  or  descending 
order  for  each  frequency  of  the  variable  grating.  The  frequencies  of  the  variable  grating  were  quasi-random.  Contrast 
threshold  measurements  were  also  determined  in  each  session.  The  data  were  collected  using  the  same  computer 
techniques  as  used  for  the  contrast  sensitivity  measurements  except  for  minor  changes  in  the  computer  programme  to 
reduce  the  number  of  trials  per  block  of  runs  and  to  accept  a  change  in  attenuation  for  each  trial. 

One  major  criticism  of  this  method  is  that  large  effects  from  adaptation  could  occur  from  free  inspection  of  high 
contrast  gratings.  However,  a  recent  study  of  contrast  matching  by  Kulikowski  (19761  has  shown  that  there  is  little  if 
any  difference  in  results  of  such  measurements  when  comparing  the  method  of  adjustment  (using  long  viewing  time)  to 
the  method  of  constant  stimuili  (using  short  viewing  time).  Furthermore,  each  setting  of  contrast  rarely  took  more  than 
10  seconds  to  make —  a  period  of  time  much  shorter  than  the  usual  2  to  3  minutes  used  for  adaptation  studies  (e.g., 
Blakemore  and  Campbell,  1969). 


A-2.4  Changing  Contrast  by  Polarizers 

A  polarizer  is  an  optical  device  whose  input  is  natural  light  and  whose  output  is  some  form  of  polarized  light.  Polarized 
light  is  light  whose  orientation  (transmission  axis  I  of  the  electric  field  is  constant  although  the  magnitude  and  sign 
vary  with  time.  Let  the  amplitude  of  the  electric  field  transmitted  by  the  polarizer  be  E0.  If  a  second  polarizer  (called 


an  analyzer)  is  placed  in  the  optical  path  of  plane  polarized  light,  only  the  component  that  is  parallel  to  the  transmission 
axis  of  the  analyzer  will  reach  the  detector  (assuming  zero  absorption  |.  This  component  will  be  equal  to  E0  cos  where 
<p  is  the  angle  of  rotation  from  the  transmission  axis.  The  irradiance,  I,  proportional  to  the  square  of  the  amplitude, 
reaching  the  detector  is  given  by  Maius's  Law,  e.g.,  see  Hecht  and  Zajac  ( 1974). 

Il$)  =  I0C0s2$ 

Note  that  at  <t>  =  90,  theoretically,  no  light  is  transmitted  because  the  electric  field  that  has  passed  through  the  polarizer 
is  perpendicular  to  the  transmission  axis  of  the  analyzer.  At  this  rotation  the  two  polarizers  are  said  to  be  crossed.  The 
resulting  electric  field  is  therefore  parallel  to  what  is  called  the  extinction  axis  of  the  analyzer  and  has  no  component 
along  the  transmission  axis.  A  more  detailed  account  of  polarization  can  be  found  in  Hecht  and  Zajac  ( 1974). 

It  is  desirable  to  have  a  simple  system  that  changes  the  contrast  of  complex  objects.  A  crossed  polarizer  system  allows  a 
controlled  amount  of  light  to  be  reflected  from  an  object.  However,  in  order  to  change  the  contrast  and  keep  the  mean 
luminance  constant,  two  crossed  Polaroid  systems  are  needed.  Constant  mean  luminance  is  achieved  if  the  light  from 
the  two  polarizers  of  orthogonal  polarization  is  collected  through  one  analyzer.  As  the  analyzer  is  rotated,  an  increase 
in  the  light  transmitted  by  one  polarizer  will  be  accompanied  by  a  decrease  in  transmission  of  the  same  amount  by  the 
other  polarizer.  The  overall  effect  of  rotating  the  analyzer  is  to  change  the  contrast  of  the  object  while  maintaining 
constant  mean  luminance. 

A  crossed  polarizing  device  was  built  with  two  projectors.  The  light  transmitted  from  the  projectors  was  directed 
through  separate  polarizers.  The  light  from  the  polarizers  was  directed  through  one  analyzer  —  a  large  circular 
piece  of  polaroid  mounted  on  a  rotating  device.  The  analyzer  angle  was  read  directly  in  degrees  from  a  scale  fixed  to  its 
mount.  The  light  from  the  two  projectors  was  imaged  onto  the  same  surface  area.  Appropriate  alignment  of  the 
polarizers  produced  maximum  and  minimum  contrast  of  the  object  as  the  analyzer  was  rotated  from  0  to  90  degrees. 

The  mean  luminance  of  the  combined  projectors  varied  ±  2  percent  as  the  analyzer  angle  rotated  from  0  to  90  degrees. 
The  mean  luminance  of  the  projected  stimuli  was  determined  using  a  linear  photocell  and  the  luminance  of  the  other 
projector  was  set  to  that  value. 

Although  the  contrast  of  an  object  could  be  calculated  directly  from  Maius's  Law,  in  practice,  any  difference  between 
the  projectors,  polarizers,  and  the  alignment  of  the  system  will  produce  errors.  Therefore,  luminance  values  were 
obtained  from  a  linear  photocell  as  the  analyzer  rotated  and  were  used  to  determine  the  contrast  versus  analyzer  angle 
curve  shown  in  Figure  96.  The  contrast  of  an  object  was  calculated  by  multiplying  the  original  contrast  of  the  object  by 
the  contrast  at  the  angle  of  the  analyzer.  Some  of  the  predicted  values  of  contrast  generally  differed  by  10  percent  from 
the  measured  values.  Thus,  it  is  quite  important  to  measure  calibration  curves  and  not  use  Maius's  Law  when 
employing  such  polariod  systems. 
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CHAPTER  A-3 


CONTRAST  SENSITIVITY  FUNCTIONS  OF  AMBLYOPES 


A-3. 1  Methods 

Contrast  sensitivity  functions  were  obtained  using  the  methods  discussed  in  Section  A-2.2.  Both  the  normal  and  the 
amblyopic  eye  were  measured  during  the  same  session.  Contrast  sensitivity  measurements  for  the  same  spatial 
frequency  taken  at  two  different  viewing  distances  were  not  averaged  to  reduce  any  possible  artifacts  in  the  analysis  of 
the  data.  Appropriate  lenses  were  placed  before  each  eye  to  provide  in-focus  gratings  with  minimum  accommodative 
effort  from  the  subject.  This  reduced  any  possible  artifacts  of  the  measurements  due  to  different  accommodative  states 
of  the  eyes  and  also  decreased  fatigue  due  to  eye  strain.  The  reference  to  the  normal  eye  is  used  only  in  a  relative  sense 
to  distinguish  the  differences  between  the  two  eyes.  It  does  not  mean  that  the  nonamblyopic  eye  is  normal. 

A-3. 2  Sensitivity  Loss  at  All  Spatial  Frequencies 

Subject  J.  P.  is  a  32-year  old  woman  with  a  minimal  left  convergent  squint,  left  hyperopia  (far-sighted  I  and  left 
amblyopia  with  normal  retinal  correspondence.  Her  Snellen  letter  acuity  was  6/5  for  the  right  eye  (R I  and  6/1  for  the 
left  eye  (LI. 

Regarding  contrast  sensitivity  shown  in  Figure  'IT,  a  general  loss  is  exhibited  in  her  amblyopic  eye  at  all  spatial 
frequencies  but  most  markedly  at  the  higher  spatial  frequencies  from  5  to  15  c/d.  The  difference  in  sensitivity  between 
the  eyes  is  more  clearly  seen  in  the  lower  curve  of  Figure  17  which  shows  a  maximum  difference  in  sensitivity  of 
about  1  IdB  at  8  to  10  c/d.  This  loss  of  sensitivity  at  all  spatial  frequencies  is  typical  of  one  class  of  amblyopes 
(Hess.  11741. 

A-3. 3  Sensitivity  Loss  at  High  Spatial  Frequencies 

A  more  severe  case  of  high  frequency  loss  was  found  in  the  amblyopic  eye  of  subject  M.B..  a  (>2-year  old  woman 
diagnosed  as  a  left  anisometropic  amblyope  with  Snellen  letter  acuity  of  6/4  (R I  and  2/60  (L).  Even  with  corrective 
lenses,  her  contrast  sensitivity  measurements,  shown  in  Figure  18.  reveal  large  differences  in  sensitivity  between  her 
amblyopic  and  her  normal  right  eye  beginning  at  2  c/d.  No  measurements  could  be  obtained  in  her  amblyopic  eye  for 
spatial  frequencies  greater  than  5  c/d.  Unfortunately,  due  to  illness  of  the  subject,  threshold  measurements  at  the  lower 
ranges  of  spatial  frequency  could  only  be  made  for  her  amblyopic  eye.  However,  the  results  obtained 
from  her  more  normal  eye  suggest  that  there  would  be  little,  if  any,  difference  between  the  eyes  at  the  lower  spatial 
frequencies.  This  general  loss  of  sensitivity  over  the  high  spatial  frequencies  is  typical  of  a  second  class  of  amblyopes 
(Hess.  11741. 

Note  that  there  is  almost  a  4  log  unit  (35  dB  I  difference  between  the  sensitivity  of  the  eyes  at  5  c/d  shown  in  the  lower 
curve  of  Figure  19.  There  appears  to  be  general  agreement  of  Snellen  acuity  and  contrast  threshold  measurements, 
indicating  a  large  degradation  in  vision  in  the  amblyopic  eye. 

Her  amblyopic  eye  is  quite  spatial  frequency  selective.  The  bandwidth  measured  at  half-sensitivity  is  about  one  octave 
with  a  centre  frequency  of  1  c/d.  In  general,  the  contrast  sensitivity  measurements  of  the  amblyopic  eye  of  this  subject 
appear  similar  to  measurements  of  a  normal  observer  with  +  3.5  diopters  of  blur  (Campbell  and  Green.  19651. 
However,  when  questioned,  the  subject  reported  the  appearance  of  objects  not  consistent  with  blur.  Although  this 
subject  was  quite  verbal  and  fully  trained  in  ophthalmic  practice,  she  could  not  convey  a  meaningful  description  of 
what  the  amblyopic  eye  saw.  She  described  colour  and  contrast  as  more  vibrant  in  her  amblyopic  eye  as  compared  to 
vision  in  her  normal  eye.  Objects  seemed  to  stand  out  from  their  background,  take  on  a  different  texture,  and  “become 
more  interesting." 

An  attempt  was  made  to  determine  if  vision  in  her  normal  eye  could  be  degraded  to  the  degree  where  objects  would 
appear  to  be  similar  in  both  eyes.  She  viewed  objects  with  her  norma)  eye  through  several  layers  of  semi-opaque  plastic 
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and  used  lenses  to  diffuse  and  blur  the  objects.  However,  even  with  appropriate  lenses  in  front  of  her  amblyopic  eye  to 
correct  for  viewing  distance,  objects  still  appeared  to  be  different  when  viewed  by  the  two  eyes.  She  could  not  equate  the 
sharpness  of  objects  between  her  two  eyes  with  induced  blur,  and  the  semi-opaque  cover  over  her  normal  eye  caused 
objects  to  look  foggy  or  misty.  These  curious  observations  were  puzzling  until  contrast  matching  studies  were  made 
with  another  amblyope.  A  discussion  of  the  possible  reasons  for  not  having  been  able  to  create  visual  conditions  in  her 
normal  eye  that  were  similar  to  those  in  her  amblyopic  eye  is  deferred  until  Section  A-4.2  where  the  results  of  the 
contrast  matching  experiments  are  discussed. 

An  attempt  to  gain  a  more  quantitative  assessment  of  her  subjective  visual  impressions  was  made  by  asking  her  to 
describe  the  filtered  portraits  shown  in  Figure  28.  She  reported  the  MTF  filtered  image  (Figure  28,a,c,2l  to  be  “better" 
portrait  than  the  original  ( Figure  28,a,c,  1 )  for  both  the  original  and  coarsely  quantized  portraits.  However,  the 
low-pass  filtered  images  of  both  the  original  and  the  coarsely  quantized  portraits  appeared  similar  to  their  MTF  filtered 
counterparts.  These  surprising  results  suggest  that  when  there  were  no  high  spatial  frequencies  present  to  provide  sharp 
detail  such  as  edges,  then  her  amblyopic  eye  saw  the  portraits  as  if  they  were  blurred.  The  high  spatial  frequency  detail 
in  the  original  portraits  appeared  to  interfere  with  perception  in  the  amblyopic  eye.  Again  these  curious  results  were 
inexplicable  until  contrast  matching  measurements  were  made  on  another  amblyope. 

A-3.4  Selective  Loss  at  Mid-Spatial  Frequencies 

A-3.4. 1  Stationary  Sine-Wave  Gratings 

The  most  interesting  and  unique  filtering  characteristics  came  from  subject  A.  H.,  a  20-year  old  female  student  who 
was  diagnosed  as  having  a  left  converging  squint,  left  hyperopia,  and  amblyopia  with  a  slight  left  eccentric  fixation  and 
abnormal  retinal  correspondence  (ARC).  Her  Snellen  acuity  was6/5  (R )  and  6/36  IL)  when  she  started  the 
experiments.  Her  initial  contrast  sensitivity  measurements  shown  in  Figure  99  reveal  large  reductions  in  sensitivity  for 
frequencies  greater  than  2  c/d.  Especially  note  that  there  is  a  considerable  decrease  in  sensitivity  of  just  over  one  log 
unit  in  her  amblyopic  eye  from  4  to  5  c/d,  suggesting  the  possibility  of  a  highly  spatial  frequency  selective  loss — a 
“notch"  at  5  c/d. 

A  second  set  of  measurements  two  days  later  produced  results  shown  in  Figure  100.  Note  especially  the  notch — the 
highly  spatial  frequency  selective  loss  at  5  c/d  suggested  from  the  initial  measurements.  The  contrast  sensitivity  in  the 
amblyopic  eye  was  increased  over  that  in  the  initial  measurements  which  resulted  in  about  a  5  dB  reduction  in 
the  difference  between  her  normal  and  amblyopic  eye  at  4  to  5  c/d.  This  increase  in  sensitivity  in  the  amblyopic  eye 
does  not  appear  to  be  due  to  a  general  practice  effect.  Note  that  the  standard  error  of  the  average  contrast  sensitivity  for 
both  eyes,  shown  in  Figure  101.  reveals  quite  stable  measurements  from  the  normal  eye, whereas  the  amblyopic 
eye  has  changed  considerably. 

The  decrease  in  the  contrast  sensitivity  difference  between  eyes  is  shown  in  Figure  102.  The  positive  data  points  indicate 
increased  sensitivity  in  the  left  eye  over  almost  the  whole  range  of  spatial  frequencies.  This  increase  in  sensitivity  in  the 
amblyopic  eye  from  the  first  to  the  second  set  of  measurements  suggests  caution  in  interpreting  the  visual  characteristics 
-  of  amblyopes  from  initial  contrast  sensitivity  measurements.  Furthermore,  such  results  suggest  the  possibility  of  a 
plasticity  in  the  mechanisms  of  amblyopia  that  are  quite  sensitive  to  stimulation.  This  increase  in  sensitivity  occurred 
after  the  amblyopic  eye  was  tested  for  only  one  hour  two  days  previously. 

Although  all  the  data  points  are  not  statistically  significant  from  one  another,  there  is  a  trend  towards  increased 
contrast  sensitivity  in  the  amblyopic  eye  at  the  low  spatial  frequencies,  below  2  c/d.  This  trend  is  evident  in  both 
Figures  Wand  100. 

The  surprising  spatial  frequency  selective  notch  in  contrast  sensitivity  is  highly  suggestive  of  depressed  activity  in  about 
a  t  one-octave  channel  with  a  centre  frequency  of  about  5  c/d.  A  review  of  the  literature  has  not  revealed  similar 
findings  except  for  data  reported  by  Itoi,  et  al.  (19761.  Three  patients  having  strabismic  amblyopia  and  one  patient 
having  ametropic  amblyopia  appear  to  have  highly  selective  losses  in  contrast  sensitivity  from  6  to  12  c/d. 

Unfortunately,  there  are  too  few  spatial  frequencies  measured  about  the  least  sensitive  spatial  frequency  to  be  certain. 
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The  notch  and  general  decrease  in  contrast  sensitivity  of  the  high  spatial  frequencies  of  subject  A.  H.  are  more  similar 
to  the  contrast  sensitivity  obtained  by  +  3.5  diopters  of  blur  by  Campbell  and  Green  1 1%51  than  that  of  subject  M.  B. 
However,  just  as  with  subject  M .  B. ,  no  amount  of  optical  defocus  of  her  normal  right  eye  could  create  vision  of  objects 
similar  to  that  seen  by  her  amblyopic  eye.  Once  again  the  reason  for  this  will  be  given  in  the  discussion  of  contrast 
matching  studies  in  Section  A -4.2. 

One  possible  cause  of  the  notch  may  be  abnormal  retinal  correspondence.  (I  am  indebted  to  Jean  Lowry  of 
Addenbrooks  Hospital.  Cambridge,  for  pointing  out  this  possibility.  I  Recall  that  ARC  occurs  when  the  peripheral 
retina  assumes  a  central  retinal  projection  instead  of  at  the  fovea  and  is  not  maintained  as  the  fixation  point  when  the 
normal  eye  is  covered.  It  is  possible  for  a  subject  to  control  the  ARC.  Dr.  Wilkins  at  Addenbrooks  Hospital, 
Cambridge,  had  a  subject  with  three  different  fixation  points  that  could  be  coi. .tolled  at  will  (personal  communication, 
1**77 1.  Such  findings  suggest  that  subject  A.  H.  could  have  been  using  different  parts  of  her  retina,  and.  hence,  different 
regions  of  spatial  sensitivity,  during  her  measurements.  For  example,  she  could  have  been  shutting  off  the  foveal  region 
during  the  measurements  of  lower  to  mid-spatial  frequencies  and  then  using  the  foveal  region  for  the  higher  spatial 
frequencies.  A  notch  in  her  contrast  sensitivity  function  would  represent  the  region  of  crossover  from  one 
retinal  field  to  the  other. 

This  hy  pothesis  was  tested  by  measuring  the  contrast  sensitivity  function  of  a  subject  that  was  not  amblyopic  but  did 
have  ARC.  The  results  shown  in  Figure  103  do  not  reveal  a  notch  in  any  region  of  the  mid  to  high  spatial  frequencies. 
This  result  suggests  that  ARC  may  not  be  the  cause  of  the  notch  in  subject  A.  H. 


A-3.4.2  Moving  Sine-Wave  Gratings 

The  next  experiments  were  designed  to  further  probe  the  tuning  characteristics  of  the  notch.  It  was  noticed  that  the 
subject  seemed  to  move  her  head  horizontally  during  the  measurements  with  her  amblyopic  eye,  as  if  to  cause  the 
grating  to  move  across  the  retina.  When  queried,  she  claimed  that,  in  general,  she  saw  the  grating  better  with  her 
amblyopic  eye  if  she  kept  it  moving  across  her  retina  by  constantly  shifting  her  gaze. 

It  ap|M-ars  that  two  types  of  mechanisms  in  the  visual  system,  transient  I Y I  and  sustained  )  X  I,  provide  separate 
channels  for  viewing  moving  and  stationary  objects  (Cleland,  Dubin.  and  Leviek,  1**7 1 ;  Rulikowski  and  Tolhurst. 

1**731.  The  behaviour  of  A.  H.,  supported  by  her  comments,  suggested  that  she  was  attempting  to  avoid  using  or  could 
not  satisfactorily  use  her  sustained  system  for  viewing  the  stationary  gratings  and  thus,  by  constantly  shifting  her  gaze, 
she  was  attempting  to  use  her  transient  system  for  the  task .  It  is  possible  that  her  tuning  characteristics  resulted  from 
partially  inoperative  sustained  mechanisms  over  the  middle  range  of  spatial  frequencies.  This  hy|>othesis  was  tested  by 
measuring  her  contrast  sensitivity  to  gratings  that  were  moved  horizontally  at  5  Hz  — a  temporal  frequency  that  provides 
maximum  sensitivity  for  normal  subjects. 

The  results  in  Figure  l(*4  show  that  the  mid-frequency  notch  has  remained  with  the  mov  ing  gratings.  Note  that  the 
contrast  sensitivity  at  the  lower  spatial  frequencies  remains  at  a  constant  value  for  both  eyes.  Such  results  are  typical 
of  the  tuning  characteristics  obtained  from  moving  gratings  in  normal  subjects  ( Robson,  l%<>;  Rulikowski  and 
Tolhurst,  1**731. 

Note  that  there  is  a  general  trend  for  an  increase  in  sensitivity  in  the  amblyopic  eye  compared  to  the  normal  eye  at  the 
lower  spatial  frequencies  in  opposition  to  the  data  obtained  with  the  stationary  gratings.  A  second  set  of  data 
collected  the  following  day  was  similar  I  Figure  1051.  The  average  of  the  two  measurements  show  n  in  Figure  100 
reveals  a  smaller  decrease  in  sensitivity  between  the  eyes  than  did  the  results  from  the  stationary  gratings,  except  at 
3  to  4  c/d.  This  smaller  decrease  in  the  difference  between  the  eyes  is  shown  in  Figure  107.  Finally,  note  that  the 
relative  contrast  sensitiv  ity  around  the  notch  at  5  c/d  is  generally  the  same  for  both  the  stationary  and  drifting  gratings, 
although  there  is  about  a  5  dB  increase  in  the  difference  curves  at  7  to  **  c/d  with  moving  gratings. 

'Fhese  results  suggest  that  the  mechanism  (or  mechaniamal  responsible  for  the  decreased  sensitivity  in  this  subject  are 
common  to  both  the  transient  and  sustained  visual  systems.  These  results  are  in  general  agreement  with  those  of  Levi 
and  Harwerth  1 1**771  who  investigated  the  spatio-temporal  interaction  of  sine-wave  gratings  in  four  subjects  that  had 
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either  ametropic  or  strabismic  amblyopia.  Similar  to  the  results  reported  here,  they  found  that  the  contrast  sensitivity 
of  the  amblyopic  and  normal  eyes  remained  the  same  at  the  higher  spatial  frequencies  but  increased  at  the  lower  spatial 
frequencies  with  flickering  gratings.  They  found,  at  the  lowest  spatial  frequency  tested  — 0.4  c/d,  that  the  contrast 
sensitivity  with  gratings  flickering  at  10  Hz  and  with  stationary  gratings  “is  essentially  equal,"  for  both  the  normal 
and  the  amblyopic  eyes  of  all  their  subjects.  However,  the  individual  measurements  of  subject  A.  H.  were  shown  to 
contain  trends  of  decreased  contrast  sensitivity  to  stationary  gratings  and  increased  contra sensitivity  to  moving 
gratings  in  the  amblyopic  eye  compared  to  the  normal  eye  at  the  low  spatial  frequencies,  1  0.2  c/d.  This  increased 

sensitivity  for  the  lower  spatial  frequencies  for  flickering  gratings  of  the  amblyopic  eye  over  the  normal  eye  is  also 
evident  in  subject  L.  N.  in  Figure  1  of  Levi  and  Harwerth,  who  may  not  be  a  strabismic  amblyope  (see  their  Table  I ). 
The  results  reported  here,  as  well  as  the  previous  analysis  of  the  data  of  Levi  and  Harwerth,  do  not  support  their 
conclusion  that  the  transient  channels  for  low  spatial  frequencies  are  normal  in  a  strabismic  or  anisometropic 
amblyopic  eye. 

Although  the  notch  of  subject  A.  H.  remained  substantially  the  same  for  both  stationary  and  moving  gratings  at  4  to  5 
c/d,  those  spatial  frequencies  are  equally  well  detected  by  both  transient  and  sustained  mechanisms  (Tolhurst,  19731. 
Significant  differences  in  that  region  of  spatial  frequency  between  stationary  and  moving  gratings  would  be  expected 
if  the  notch  were  caused  by  either  the  transient  or  sustained  mechanisms.  Thus,  the  results  reported  here,  coupled  with 
the  previous  analysis,  suggest  that  both  mechanisms  may  be  involved  in  strabismic  amblyopia.  However,  further 
research  is  needed  to  determine  the  relative  importance  of  each  mechanism  in  the  overall  loss  of  spatial  vision  that  an 
amblyope  suffers. 

A-3.4.3  Sine-W  ave  Versus  Square- W  ave  Gratings 

The  next  step  in  this  investigation  was  to  determine  whether  a  difference  in  contrast  sensitivity  existed  for  subject  A.H. 
between  sine-wave  and  square-wave  gratings.  Campbell  and  Robson  ( 19681  showed  that  above  1  c/d  the  threshold  of  a 
square-wave  grating  is  1 .27  (4/  TT  I  greater  than  that  of  a  sine-wave  grating.  This  result  was  predicted  from  an 
assumption  of  linearity  at  threshold  and  that  detection  at  threshold  was  based  upon  the  energy  of  the  Fourier 
components  of  the  square  wave. 

The  results  of  the  contrast  sensitivity  measurements  of  square-wave  gratings  averaged  over  three  sessions  are  shown  in 
Figure  108.  Note  that  the  mid-frequency  notch  that  was  found  using  stationary  and  moving  gratings  has  almost 
vanished.  The  peak  difference  in  sensitivity  of  about  20  dB  between  the  amblyopic  and  the  normal  eye  has  shifted  from 
4-7  c/d  to  7- 10  c/d.  This  increased  sensitivity  in  the  mid-frequency  notch  was  quite  surprising. 

This  result  suggested  that  there  was  a  difference  in  sensitivity  to  sine-  and  square-wave  gratings  which  was  tested  by 
measuring  the  contrast  sensitivity  using  square  and  sine  waves  during  the  same  session.  As  can  be  seen  from  the  results 
in  Figure  109,  little  difference  between  contrast  sensitivity  to  sine-  and  square-wave  gratings  was  found.  In  general,  the 
square  waves  were  detected  at  a  lower  contrast  than  the  sine  waves,  which  is  in  general  agreement  with  the  results 
obtained  by  Campbell  and  Robson  ( 19681.  A  closer  agreement  with  the  4/  7 T  difference  between  the  contrast  sensitivity 
of  square-  and  sine-wave  gratings  is  lacking,  presumably  due  to  the  observer’s  lack  of  experience  in  performing  such 
tasks  and  also  the  fact  that  the  presentations  of  the  sine  and  square  waves  were  not  alternated  at  each  frequency  for 
each  measurement. 

<  Inly  the  high  spatial  frequencies  were  measured  in  Figure  109  and  for  the  subsequent  contrast  sensitivity 
measurements  due  to  the  short  time  that  the  subject  was  available.  She  was  usually  able  to  be  a  subject  for  only  two 
hours  per  week.  Since  the  major  changes  in  her  contrast  sensitivity  function  were  at  the  high  spatial  frequencies,  the 
exclusion  of  low  spatial  frequencies  allowed  more  time  to  be  used  for  the  following  contrast  matching  experiments. 

A-3.3  Changes  in  Contrast  Sensitivity  in  the  Amblyopic  Kye 

The  increased  sensitivity  from  4  to  7  c/d  and  the  concomitant  shift  to  a  higher  spatial  frequency  of  the  peak  difference 
between  the  eyes  has,  in  general,  confirmed  the  previous  results  with  square  waves.  The  difference  in  contrast 
sensitivity  between  her  eyes  from  her  initial  measurements  to  this  last  set  is  shown  in  Figure  1 10.  There  is  over  a  ten-fold 
the  increase  in  the  difference  in  sensitivity  at  4  c/d;  this  decreases  to  less  than  half  that  value  at  6  c/d!  Large  changes  in 
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sensitivity  over  such  a  narrow  range  of  spatial  frequencies  suggest  a  certain  plasticity  in  highly  tuned  one-to*two-octave 
bandwidth  mechanisms  in  the  visual  system. 

Could  such  a  change  be  due  to  artifacts  such  as  misalignment  or  changes  in  the  apparatus?  Such  factors  can  be  ruled 
out  because  if,  for  example,  the  viewing  distance  was  wrong  or  the  oscillator  malfunctioned,  then  general  differences  in 
sensitivity  would  be  expected  at  all  frequencies  — not  at  those  just  around  4  c/d.  However,  the  contrast  sensitivity 
measurements  below  2  c/d  and  above  10  c/d  remained  substantially  the  same.  Furthermore,  such  artifacts  should 
produce  concomitant  changes  in  the  sensitivity  measurements  in  the  normal  eye.  Since  there  is  no  evidence  for  such 
changes  in  the  normal  eye,  it  was  assumed  that  the  change  in  sensitivity  in  her  amblyopic  eye  reflected  a  real  change  in 
the  state  of  her  visual  system. 

The  increase  in  sensitivity  to  the  mid-range  of  spatial  frequencies  in  the  amblyopic  eye  should  be  reflected  in  another 
measure  of  acuity.  Thus,  the  subject  was  tested  for  Snellen  letter  acuity  in  which,  surprisingly,  the  amblyopic  eye 
which  was  originally  tested  at  6/36  now  tested  at  6/18  —  a  gain  in  acuity  of  two  lines.  Upon  testing,  the  subject,  who  had 
been  tested  with  Snellen  charts  many  times  in  the  past,  exclaimed,  “I  have  never  seen  those  lines  before  with  my  bad 
eye. "  A  more  detailed  analysis  that  relates  this  subject's  change  in  Snellen  acuity  to  changes  in  contrast  sensitivity  can  be 
found  in  Section  18.3. 

In  one  sense,  this  subject's  increase  in  Snellen  letter  acuity  should  not  be  too  surprising.  Quite  striking  increases  in  the 
performance  of  the  visual  systems  of  amblyopes  have  been  reported  by  many  researchers,  e.g..  Von  Noorden  ( 19671. 
Furthermore.  Itoi,  et  al.  ( 19751.  also  found  increased  contrast  sensitivity  with  two  amblyopes  after  40  days  and  two 
months  occlusion  of  the  normal  eye.  Their  data  showed  an  increase  predominantly  in  the  middle  spatial  frequencies. 

The  earlier  notches  were  not  present  after  occlusion.  However,  for  some  unexplained  reason,  the  second  measurements 
were  made  at  spatial  frequencies  other  than  at  the  notch,  and  it  is  uncertain  whether  their  results  are  the  same  as  those 
reported  here. 

The  improved  performance  in  their  cases  was  due  to  specific  forms  of  therapy,  e.g..  occlusion.  In  other  words, 
increased  acuity  occurred  in  certain  types  of  amblyopes  by  working  the  heretofore  suppressed  amblyopic  eye.  What  is 
interesting  about  the  results  reported  here  is  that  there  was  no  conscious  attempt  to  improve  the  sight  in  the  amblyopic 
eye.  Rather,  the  increased  acuity  was  an  inadvertent  result  of  having  the  subject  use  the  abnormal  eye  for 
comparatively  difficult  threshold  measurements. 

It  would  seem  that  the  time  span  over  which  these  results  occurred,  as  well  as  the  amount  of  time  the  subject  spent  in 
the  experiment,  is  important.  Her  amblyopic  eye  was  used  only  four  times  for  about  one  hour  each  time  over  a  period  of 
one  week  for  the  collection  of  the  data  in  the  initial  sensitivity  measurements  using  stationary  and  moving  gratings.  A 
four- month  break  followed  due  to  a  summer  holiday.  Upon  her  return,  the  contrast  sensitivity  of  the  subject  was 
determined  using  square-wave  gratings,  after  which  the  combined  sine-  and  square-wave  measurements  were  made  and 
Snellen  letter  acuity  was  remeasured.  This  increase  in  visual  performance,  if  it  can  be  replicated  with  other  amblyopes. 
suggests  that  gratings  may  offer  a  powerful  tool,  not  only  in  the  assessment  of  amblyopia  but  as  a  possible  means  of 
therapy.  In  particular,  it  may  be  that  only  short  exposure  to  gratings  that  are  continuously  shifted  in  spatial  frequency 
might  prove  quite  useful.  Such  patterns  would  stimulate  all  visual  channels  equally  well. 


The  evidence  of  a  substantial  shift  in  a  relatively  narrow  region  of  spatial  frequency  with  time  suggested  that  it  would 
be  of  interest  to  measure  A.  H.  periodically.  Since  she  was  about  to  go  on  an  extended  holiday  of  about  one  and  one  half 
months,  another  session  was  planned  just  before  her  departure.  The  results  of  the  contrast  sensitivity  measurements 
made  prior  to  her  departure  showed  a  slight  decrease  of  contrast  sensitivity  in  the  amblyopic  eye.  The  peak  difference 
between  eyes  was  about  20  dBfrom  6  to  7  c/d  (see  Figure  1111.  This  change  was  not  considered  to  bo  significant  until 
the  data  that  were  obtained  from  measurements  made  after  her  return  were  found  to  be  quite  similar,  as  shown  in  Figurp 
1 12.  Here,  a  peak  difference  in  sensitivity  of  17.5  dRwas  noted.  The  final  measurements  two  months  later  are  shown  in 
Figure  1 13.  The  peak  difference  in  sensitivity  is  16  dB  at  6  to  7  c/d.  Although  there  is  evidence  for  a  slight  decrease  in 
the  peak  difference  between  eyes  in  spatial  frequency,  there  was  a  consistent  reduction  of  about  4  dBin  these  last  three 
sets  of  measurements  as  compared  to  previous  ones. Her  Snellen  acuity  remained  at  6/18.  This  subject  was  retested  for 
contrast  sensitivity  and  Snellen  acuity  after  10  months.  No  significant  decrease  in  contrast  sensitivity  or  Snellen 
acuity  had  ••cciirred . 
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It  can  be  concluded  that,  in  general.  A.  H.  's  loss  of  the  highly  selective  spatial  frequency  notch  has  remained  relatively 
stable  for  a  period  of  about  one  and  one-half  years. 

The  peak  contrast  sensitivity  of  subject  A.  H.  at  2  c/d  increased  considerably  in  both  her  normal  and  amblyopic  eye  over 
the  34-week  period  when  measurements  were  made.  This  trend  of  increased  sensitivity,  shown  in  Figure  1 14.  is  seen 
from  the  straight  lines  that  have  been  drawn  by  eye.  Although  both  eyes  have  increased  in  sensitivity  during  the  course 
of  the  experiments,  the  normal  eye  has  increased  at  a  higher  rate  than  the  amblyopic  eye.  Further  research  is 
needed  to  determine  if  the  increased  sensitivity  resulted  from  practice  effects  and/or  if  the  physiological  sensitivity 
increased.  Similar  results  of  increased  contrast  sensitivity  from  normal  subjects  tested  for  over  one  year  have  been  noted 
by  DeValois  1 10771. 

A-3.6  Summary 

The  results  of  determining  contrast  sensitivity  functions  of  several  amblyopes  have  been  shown.  One  amblvope  showed 
a  unique  highly  spatial  frequency  selective  region  of  decreased  contrast  sensitivity  at  mid-spatial  frequencies.  Th  is 
region  of  decreased  sensitivity  showed  an  increase  of  over  20  dB  from  initial  to  final  measurements.  These  novel  results 
are  concerned  with  threshold  vision.  The  next  study  involved  investigation  of  the  suprathreshold  vision  of  the  same 
amblvope. 
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CHAPTER  A-4 


SUPRATHRESHOLD  VISION  OF  AN  AMBLYOPE 


An  attempt  was  made  to  make  9ine  and  square  waves  look  similar  to  the  amblyopic  eye  of  subject  A.H.  by  changing 
contrast.  No  systematic  range  of  contrast  produced  consistent  waveforms  that  she  would  call  sine  or  square  wave  over 
the  range  of  spatial  frequencies  tested  —  1  to  7  c/d.  Furthermore,  an  attempt  was  made  to  determine  whether  square 
waves  could  be  seen  as  the  same  waveform  in  both  eyes  by  changing  only  contrast.  Again  no  systematic  results  were 
obtained.  These  negative  results  suggested  a  very  complex  interrelationship  between  contrast  and  spatial  frequency. 

One  interesting  result  of  the  investigations  of  amblyopes  was  the  inability  of  the  subjects  to  describe,  except  in  the  most 
general  terms,  what  they  saw  with  their  amblyopic  eye.  This  result  i9  curious  because  each  had  a  relatively  normal 
eye  with  which  to  compare  vision  in  the  amblyopic  eye.  Yet  no  optical  means  used  to  reduce  acuity  and/ or  contrast  in 
the  normal  eye  seemed  to  provide  objects  that  appeared  similar  to  those  seen  by  the  amblyopic  eye.  These  results  were 
surprising  since  the  contrast  sensitivity  functions  in  two  of  the  amblyopes  were  quite  similar  to  those  obtained  with 
optical  blur,  as  previously  discussed. 

There  is,  however,  another  factor  to  consider  in  attempts  to  equate  the  perception  of  suprathreshold  objects  that  are 
composed  of  complex  spatial  frequency  distributions  to  the  perception  of  spatially  pure  objects  such  as  gratings  at 
threshold.  Objects  have  many  different  spatial  frequencies  in  two  dimensions  at  many  different  levels  of  contrast.  The 
results  of  contrast  matching  experiments  which  have  been  discussed  suggest  that  the  gain  of  mechanisms  tuned  to 
different  ranges  of  spatial  frequencies  is  at  different  levels  of  activity,  depending  upon  contrast.  Although  the  contrast 
matching  functions  are  a  well  behaved  family  of  functions  (similar  monotonic  increasing  and  decreasing  functions!  for 
the  normal  observer  (see  Figure  131,  there  may  be  differences  in  contrast  matching  between  a  normal  and  amblyopic 
eye.  And.  such  differences  may  account  for  the  strange  and  poor  descriptions  of  objects  seen  with  an  amblyopic  eye. 

Such  considerations,  as  well  as  the  desire  to  gather  more  quantitative  data  on  amblyopia  using  a  technique  that  has  not 
been  used  before  on  amblyopes,  prompted  the  next  set  of  experiments — contrast  matching. 

A-4. 1  Contrast  Matching  of  a  Normal  Observer 

Contrast  matching  data  from  a  corrected  astigmat,  A.  G..  the  author,  were  obtained  in  order  to  provide  a  reference  for 
similar  data  that  were  collected  from  the  amblyope  A.  H.  The  methods  used  were  discussed  in  A-2.3.  Figure  13  shows 
that  the  contrast  matching  data  of  subject  A.  G.  provide  a  family  of  equal-contrast  contours  plotted  similarly  to  the  data 
of  Georgeson  and  Sullivan  (19751.  The  standard  spatial  frequency  U9ed  was  2  c/d.  the  spatial  frequency  for  which 
contrast  sensitivity  was  maximal  in  previous  measurements.  The  standard  contrasts  at  the  standard  spatial  frequency- 
are  represented  by  the  triangular  symbols.  The  connected  open  symbols  represent  the  contrast  of  the  test  grating  that 
appeared  to  match  the  contrast  of  the  standard  grating.  A  physical  match  between  the  standard  and  the  test  grating  will 
result  in  equal  contrast  curves  which  are  horizontal  lines  through  the  standard  contrast  symbols.  Such  results  are 
evident,  in  general,  from  0.2  to  10  c/d  for  the  higher  contrasts.  However,  at  lower  standard  contrasts  the  equal-contrast 
curves  approach  the  contrast  sensitivity  function.  The  uppermost  curve  is  the  contrast  sensitivity  function. 

These  results  are  in  general  agreement  with  similar  measurements  of  other  researchers,  e.g..  Watanabe,  et  al.  ( 19681; 
Georgeson  and  Sullivan  1 19751. 

A-4. 2  Contrast  Matching  by  an  Amblyope 

The  contrast  matching  data  for  the  amblyope  A.  H.  are  shown  in  Figure  1 1 5.  The  low  spatial  frequency  contrast 
matching  data  from  0.2  to  2  c/d  were  based  on  the  average  of  five  trials.  The  high  spatial  frequency  contrast  matching 
data  from  2  to  20  c/d  were  based  on  the  average  of  three  sessions  of  five  trials  each.  As  with  the  contrast  sensitivity 
measurements,  the  normal  and  amblyopic  eye  were  tested  during  the  same  session. 

The  contrast  matches  for  the  low  spatial  frequencies  appear  normal  when  compared  to  the  results  of  subject  A.  G. 


( Figure  13 1.  The  contrast  measurement  and  contrast  match  at  the  low  levels  of  contrast  were  as  expected,  showing  a 
mid-frequency  loss  and  cutoff.  The  highese  curve  is  A.  H.’s  contrast  sensitivity  function.  However,  the  results  at  high 
contrast  were  quite  unexpected  for  both  the  normal  and  the  amblyopic  eye.  The  loss  of  sensitivity  of  the  amblyopic  eye 
shown  by  matches  at  5  c/d  at  contrasts  below  16.5  percent  is  not  evident  at  the  high  contrasts  of  33  percent  and  M> 
percent.  There  is  also  a  trend  of  increased  sensitivity  at  10  c/d,  especially  at  the  33  percent  contrast  level.  Note  that  this 
increased  sensitivity  at  10  c/d  reverses  at  15  c/d. 

The  contrast  matches  in  the  normal  eye  were  as  expected  except  at  10  c/d  where  an  abnormal  decrease  in  the  ability  to 
make  a  contrast  match  was  evident  in  general  for  higher  contrasts  of  33  percent  and  66  percent.  Such  losses  at  10  c/d 
at  the  higher  contrasts  are  not  found  in  the  contrast  matches  of  subject  A.  G.  Thus.  A.  H.  's  "normal"  eye  may  not  be 
normal.  These  quite  unexpected  contrast  and  spatial  frequency  selective  differences  from  the  contrast  match  in  her 
normal  and  amblyopic  eye  are  reflected  in  each  of  the  three  testing  sessions  as  shown  in  Figure  1 16. 

The  individual  and  average  differences  in  contrast  matches  between  the  normal  and  amblyopic  eye  are  seen  more 
clearly  in  the  difference  curves  of  Figures  117  and  1 18.  The  differences  in  sensitivity  between  the  normal  and  the 
ambly  opic  eye  are  expressed  in  decibels.  The  numbers  on  the  left  side  of  each  set  of  curves  are  the  standard  contrasts. 
The  three  sets  of  measurements  in  Figure  1 17  are  from  three  different  test  sessions.  No  data  are  shown  for  the  standard 
contrast  of  0.01 1  on  the  16/3/77  or  22/2/77  test  session  because  the  subject  could  not  do  a  contrast  match  at  the  0.01 1 
standard  contrast.  Thus,  the  results  shown  for  the  0.021  standard  contrasts  for  those  measurements  are  the  contrast 
sensitivity  measurements. 

Note  the  increased  sensitivity  of  the  amblyopic  eye  at  10  c/d  at  high  contrast  and  the  decreased  sensitivity  at  5  c/d  at 
low  contrast  w ith  respect  to  the  other  eye.  Positive  differences  in  Figures  1 1 7  and  1 18  mean  that  the  normal  eye 
matched  more  closely  to  the  baseline  contrast  match  at  2  c/d  than  the  amblyopic  eye.  Negative  differences  reflect  the 
reverse  situation.  The  average  differences  between  eyes  in  Figure  1 18  are  the  averages  of  the  differences  of  all  the 
measurements.  The  vertical  lines  indicate  plus  and  minus  two  standard  errors.  This  method  of  determining  the  average 
differences  is  only  valid  if  the  individual  differences  are  of  the  same  sign,  if  some  individual  differences  are  of  opposite 
sign,  then  the  numerical  average  will  not  reflect  the  degree  to  which  the  two  eyes  differ.  This  situation  occurred 
primarily  with  the  high  standard  contrast  of  66  percent.  Therefore,  the  average  values  for  the  differences  between  eyes 
for  the  standard  contrast  of  66  percent  were  determined  from  the  absolute  values  of  the  individual  differences.  The  sign 
was  then  chosen  to  reflect  the  eye  which  had  the  closest  contrast  matches  to  the  baseline  contrast  match  at  2  c/d.  For 
example,  an  examination  of  the  individual  differences  between  eyes  in  Figure  1 16  showed  that  the  amblyopic  eye 
performed  a  contrast  match  better  than  the  normal  eye  at  10  c/d.  This  result  would  not  be  evident  in  the  average 
differences  had  not  the  latter  analysis  been  used. 

These  surprising  results  can  be  seen  in  greater  detail  in  the  additional  contrast  matching  data  that  were  obtained  over  a 
more  concentrated  range  of  mid-spatial  frequencies.  The  contrast  matches  of  two  sessions  from  2  to  10  c/d  in  2  c/d 
increments  are  shown  in  Figures  1 10, a  and  1 10, b.  The  average  results  are  shown  in  Figure  1 10, c.  The  difference 
curves  are  shown  in  Figure  120.  Results  similar  to  the  previous  measurements  are  evident.  The  decrease  in  sensitivity 
most  evident  at  band  8  c/d  at  low  contrast  is  not  found  at  contrasts  of  33  percent  and  66  percent.  Her  amblyopic  eye 
shows  increased  sensitivity  at  33  and  66  percent  contrast.  As  in  the  previous  results,  her  normal  eye  has  lost  sensitivity 
at  10  c/d  for  the  high  contrasts.  Clearly,  the  state  of  both  her  normal  and  amblyopic  eye  changes  as  a  function  of 
spatial  frequency  and  contrast. 

Note  that  there  is  a  greater  difference  between  the  contrast  match  at  the  higher  standard  contrasts  in  the  first  set  of  data 
iFigure  120,a)  as  when  compared  to  the  second  set  (Figure  120,bl.  The  second  set  of  data  is  similar  to  the  first  set 
primarily  at  lower  contrasts.  Almost  two  months  elapsed  between  these  particular  measurements.  It  can  only  be 
conjectured  whether  these  results  are  due  to  practice  or  whether  they  reflect  a  change  of  state  of  her  visual  system  such 
as  occurred  with  the  previous  contrast  sensitivity  and  Snellen  letter  acuity  measurements  or  normal  variability. 
Further  investigation  is  needed  before  any  determination  of  such  changes  can  be  made. 


A-4.3  Discussion 

These  results  show  that  the  mechanisms  tuned  to  different  spatial  frequencies  in  the  visual  system  of  this  amblyope  do 
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not  behave  normally.  Mechanisms  lor  both  the  amblyopic  and  normal  eye  changed  their  sensitivity  in  a  complex 
manner  as  a  function  of  spatial  frequency  and  contrast.  The  results  of  the  contrast  matches  at  low  contrasts  were 
consistent  with  the  previous  contrast  sensitivity  data.  However,  at  high  contrast  the  amblyopic  eye  appeared  to 
overcompensate,  requiring  less  contrast  than  should  be  required  to  do  a  contrast  match.  The  normal  eye  at  higher 
contrast  appeared  to  undercompensate  and  required  more  contrast  than  was  needed  in  comparison  to  the  normal 
subject.  Although  the  previous  data  from  contrast  matching,  e.g.,  Georgeson  and  Sullivan  1 19751,  can  be  interpreted  as 
evidence  for  mechanisms  in  the  visual  system  tuned  to  different  ranges  of  spatial  frequencies,  the  present  data  obtained 
from  this  amblvope  most  strongly  demonstrated  visual  behaviour  that  is  difficult  to  interpret  in  any  other  way.  Further¬ 
more,  the  overcompensation  and  undercompensation  that  were  found  suggest  a  much  more  complex  gain  system  for  the 
channels  than  was  suggested  by  Georgeson  and  Sullivan  11975). 

Although  further  research  is  needed  to  determine  the  rules  of  such  channel  behaviour,  certain  speculation  can  be  made 
about  possible  reasons  for  overcompensation  in  the  amblyopic  eye  and  undercompensation  in  the  normal  eye.  As  noted 
in  the  introduction,  the  amblvope  has  two  problems  to  overcome:  confusion  and  diplopia.  The  amblyope  can 
sometimes  maintain  binocular  vision.  A.  H.  does  have  limited  stereopsis.  Perhaps  the  overcompensation  in  her 
amblyopic  eye  and  the  undercompensation  in  her  normal  eye  reflect  a  need  of  the  visual  system  to  maintain  constant 
contrast  for  binocular  mechanisms  in  a  manner  similar  to  a  binocular  brightness  averaging  mechanism  proposed  by 
Levelt  1 19651. 

The  gain  of  the  least  sensitive  channel  in  the  amblyopic  eye  may  have  been  boosted  (reduced  inhibition )  to  compensate 
for  the  low  sensitivity.  If  the  gain  mechanism  had  a  similar  or  larger  bandwidth  than  that  of  the  channels,  then  a  boost 
of  gain  of  one  channel  would  cause  a  boost  of  gain  in  adjacent  channels.  Since  the  channels  are  about  one  to  two  octaves 
wide,  an  increase  in  sensitivity  for  the  6  c/d  channel  would  also  cause  a  boost  in  higher  and  lower  spatial  frequencies, 
resulting  in  overcompensation,  as  was  found  at  10  c/d.  Although  this  explanation  seems  reasonable,  it  does  not  help 
explain  why  the  channels  overcompensate  at  higher  levels  of  contrast  or  why  the  channels  at  3  c/d  do  not  also  show 
overcompensation.  However,  suppose  that  the  gain  of  certain  channels  in  the  amblyopic  eye  was  boosted  and  resulted 
in  abnormally  high  sensitivity  at  certain  levels  of  contrast.  Then  how  is  the  contrast  of  these  channels  combined  with 
the  contrast  of  the  channels  in  the  other  eye  if,  for  example,  an  average  contrast  must  be  maintained?  One  solution 
would  be  a  decrease  in  the  gain  of  certain  channels  in  the  normal  eye  to  compensate  for  the  increased  gain  in  the 
amblyopic  eye. 

This  idea  of  a  channel  having  abnormally  low  sensitivity  could  be  reinterpreted  in  termsof  a  channel  having 
abnormally  high  sensitivity.  If  the  gain  mechanism  for  the  10  c/d  channel  were  abnormally  high,  it  could  reduce 
the  output  of  other  channels  by  inhibition.  This  explanation  fits  the  data  better  because  there  did  not  seem  to  be  any 
overcompensation  at  3  c/d  in  the  amblyopic  eye  and  it  is  more  likely  (e.g.,  Harmon  and  Julesz,  1973)  that  high 
spatial  frequencies  inhibit  low  spatial  frequencies  than  that  low  spatial  frequencies  inhibit  high  spatial  frequencies.  The 
rest  of  the  argument  follows  as  above.  At  present,  these  explanations  are  quite  speculative.  However,  the  contrast 
matching  data  clearly  imply  much  more  complex  interactions  between  the  normal  and  amblyopic  eye  than  has  been 
previously  suggested. 

If  these  quite  unexpected  results  can  be  found  with  other  amblyopes.  then  perhaps  a  beginning  can  be  made  to 
understand  the  difficulty  that  amblyopes  (and  possibly  other  individuals  with  different  visual  disorders!  have 
in  describing  the  characteristics  of  objects  that  they  see.  Furthermore,  these  results  also  suggest  why  no  optical 
means  could  produce  images  in  the  normal  eye  that  looked  like  the  images  seen  with  the  amblyopic  eye.  Most  objects 
contain  many  spatial  frequencies  at  different  contrast  levels.  The  contrast  matching  experiments  have  shown  that  the 
sensitivity  of  the  mechanisms  in  the  visual  system  of  this  amblyope  depends  not  only  on  the  spatial  frequency 
composition  but  also  th.-  contrast  of  spatial  frequencies.  It  is  not  surprising  that  optical  or  other  techniques  which  affect 
contrast  at  different  spatial  frequencies  in  a  monotonic  fashion  cannot  mimic  the  vision  of  an  amblyope.  The  visual  system 
of  an  amblyope  appears  to  provide  a  much  more  complex  picture  of  objects  than  has  been  suspected — a  picture  that,  as  one 
subject  stated,  is  “more  interesting. ” 

Although  it  appears  that  the  contrast  matching  studies  help  to  explain  the  strange  suprathreshold  vision  of  the 
amblyope,  there  is  another  question  to  be  answered  from  the  original  contrast  sensitivity  measurements —  is 
there  a  relationship  between  the  increased  spatial  frequency  sensitivity  in  the  amblyopic  eye  and  the  concomitant 
increase  in  Snellen  letter  acuity?  This  is  examined  in  Section  18.3. 


CHAPTER  A-5 


THE  FILTERING  CHARACTERISTICS  OF  PATIENTS 
HAVING  MULTIPLE  SCLEROSIS 


A-5.1  Introduction 

Multiple  sclerosis  (MSI  is  a  disease  which  results  from  demyelination  of  conducting  fibres  in  the  nervous  system. 
Demyelination  in  the  visual  system,  typically  of  the  optic  nerve  or  chiasma,  is  almost  always  found  postmortem  in  cases 
of  MS  (Duke-Elder,  1949,  Vol.  Ill;  Lumsden,  1970).  However,  it  is  quite  common  for  patients  suffering  from  MS  to 
complain  of  poor  vision  in  one  or  both  eyes  and  yet  have  normal  Snellen  acuity.  Such  reports  suggest  that  MS  patients 
may  have  selective  losses  in  spatial  frequency  channels  that  are  not  sufficiently  low  to  affect  the  range  of  spatial 
frequencies  required  for  Snellen  letter  acuity  but  that  do  affect  the  general  quality  of  vision.  Motivated  by  the  study 
of  the  abnormal  filtering  characteristics  of  amblyopia,  a  similar  investigation  was  made  with  patients  having  MS. 

A-5. 2  Methods  and  Results 

The  contrast  sensitivity  functions  of  six  MS  patients  were  determined  using  the  same  methods  discussed  previously  in 
Section  A-2.  The  results  are  shown  in  Figures  121  to  126.  The  visual  acuity  of  the  subjects  is  shown  in  the  upper  right 
corner  of  each  figure.  The  difference  in  sensitivity  between  eyes  is  plotted  below  the  contrast  sensitivity  curves,  in 
factors  on  the  left,  dB  on  the  right.  Compared  with  the  contrast  sensitivity  curves  of  the  amblyopes.  there  is  generally 
little  difference  in  sensitivity  between  the  two  eyes  at  the  high  spatial  frequencies.  However,  each  patient  shows  at  least 
a  7 dB  (greater  than  a  factor  of  2)  difference  in  sensitivity  between  eyes  over  some  range  of  spatial  frequencies.  Three 
general  classes  of  sensitivity  loss  are  evident;  mid-spatial  frequencies  (subjects  C.  H.,  A.  J.,  M.  D.),  all  spatial 
frequencies  (subject  S.  S..  M.  S.l,  high  spatial  frequencies  (subject  B.  M.l.  One  subject,  C.  H.,  showed  an  average  of  3 
dB  increased  sensitivity  from  0.2  to  0.3  c/d  in  the  left  comparaed  to  the  right  eye.  However,  at  4  to  10  c/d,  the  right  eye 
was  about  14  dB  more  sensitive  than  the  left  eye. 

A-5. 3  Discussion 

Clearly,  the  contrast  sensitivity  measurements  provide  a  much  more  sensitive  indication  of  the  filtering  characteristics 
of  the  visual  system  that  is  used  for  the  everyday  perception  of  objects  than  does  Snellen  letter  acuity.  The  predominant 
mid-frequency  losses  in  sensitivity,  especially  found  with  subjects  C.  H..  A.  J..  and  M.  D.,  would  be  very  difficult  to 
explain  by  simple  optical  blurring  which  always  degrades  the  high  spatial  frequencies  more  than  the  low.  However, 
these  results  are  consistent  with  selective  losses  in  channels  tuned  to  different  ranges  of  spatial  frequency. 

These  results  are  also  in  good  agreement  with  a  similar  recent  study  by  Regan.  Silver,  and  Murray  ( 1977).  They  found 
that  20  of  48  patients  had  losses  in  contrast  sensitivity  that  could  not  be  explained  by  refractive  error.  Sensitivity  losses 
in  the  mid-spatial  frequencies  were  evident  in  10  of  20  patients.  Three  patients  were  reported  to  have  selective  losses  at 
low-  to  mid-spatial  frequencies.  Since  8  of  14  patients  with  selective  spatial  frequency  losses  did  not  show  any  clinical 
evidence  of  visual  impairment,  they  concluded  that  the  contrast  sensitivity  measurements  can  aid  the  earlier  diagnosis 
of  MS. 

The  selective  spatial  frequency  losses  in  contrast  sensitivity  found  to  accompany  MS  have  not  resulted  in  the  large 
losses  of  Snellen  letter  acuity  that  were  seen  with  the  amblyopes.  The  only  general  difference  between  the  contrast 
sensitivity  functions  of  MS  and  amblyopia  is  the  degree  to  which  the  contrast  sensitivity  is  decreased  over  certain  ranges 
of  spatial  frequency.  This  finding  further  suggests  that  Snellen  letter  acuity  is  based  upon  a  certain  level  of  contrast 
sensitivity  over  a  relatively  narrow  range  of  high  spatial  frequencies. 

The  level  of  contrast  sensitivity  attained  by  these  MS  patients  to  their  Snellen  letter  acuity  is  related  in  Section  18.2. 


APPENDIX  B 

TABLE  OF  CONTRAST  SENSITIVITY 
FROM  CAMPBELL.  KULIKOWSK1,  AND  LEVINSON  (19661 


B-l 


Q 

6^  U 
Ptf 


CM^in(NHH'}LOOtSO'(MD'jOOOinin 

HNnmvDMO'OOM^inNHHHOOO 


*»'•**-"  *- 


APPENDIX  C 

OBJECTS  THAT  ARE  BLURRED  BY  OPTICAL  DEFOCUS 
TO  ABOUT  THE  FOURTH  HARMONIC 
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nequal  Sized  Fins  Going  in  the  Same  Direction:  Ipl  Benussi  Ring 
Ir  I  Miiller- Lyer  Illusion  with  Fins  Only  with  Further  Blurring; 
Blurring:  (ul  Rubin  Face- Vase  Object,  (cont.l 


Similarity :  (d)  Dot-Line-Proximity /Similarity;  le)  Dot  Letter  G;  If)  Dot  Letter  R  in  Noise;  Igl  Letter  E; 
ine  Letter  H;  <il  Isolating  a  Rectangular  Form  from  Oblique  Lines;  Ijl  Poggendorff  Illusion;  <kl  Zollner 


